connect.lonelyplanet.com
robots.txt

Robots Exclusion Standard data for connect.lonelyplanet.com

Resource Scan

Scan Details

Site Domain connect.lonelyplanet.com
Base Domain lonelyplanet.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-05-10T16:21:18+00:00
Next Scan 2024-08-08T16:21:18+00:00

Last Successful Scan

Scanned2023-01-08T16:45:21+00:00
URL https://connect.lonelyplanet.com/robots.txt
Domain IPs 151.101.0.143, 151.101.128.143, 151.101.192.143, 151.101.64.143
Response IP 199.232.44.143
Found Yes
Hash a4a7ad8d073291cf7033f0bd5624537d4e7593add4c2baed6d61b08692c52b04
SimHash 2de25c30e382

Groups

*

Rule Path
Disallow /gateway/
Disallow /thorntree/*
Disallow /members/*
Disallow /assets/
Disallow /*?create_review=true*
Disallow */v/vid/*?tag
Disallow /*/articles/category/*
Disallow *?type
Disallow *?sort
Disallow *?subtype
Disallow *?neighborhoods
Disallow */pa-act/
Disallow */pa-tou/
Disallow */poi-sho/
Disallow */poi-ent/
Disallow */poi-sle/
Disallow */poi-dri/
Disallow */poi-eat/
Disallow *searchResult?q
Disallow *search?q
Disallow *?utf8
Disallow *?search
Disallow *?&search
Disallow *%26search
Disallow *?sort_dir
Disallow *?sort_order
Disallow *?filters
Disallow *?ab_tours_cta
Disallow *?authenticity_token
Disallow *?ab_viator_activity_cta
Disallow *?lpaffil
Disallow *?category

sphere (www.sphere.com) scout*at*sphere*dot*com

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 0.5

blexbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

blekkobot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

voltron

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.lonelyplanet.com/articles/sitemap.xml
sitemap https://www.lonelyplanet.com/video/smap.xml
sitemap https://www.lonelyplanet.com/articles/google-news
sitemap https://www.lonelyplanet.com/stories/sitemap-index.xml

Warnings

  • 2 invalid lines.