cntravellerme.com
robots.txt

Robots Exclusion Standard data for cntravellerme.com

Resource Scan

Scan Details

Site Domain cntravellerme.com
Base Domain cntravellerme.com
Scan Status Ok
Last Scan2024-09-23T16:01:18+00:00
Next Scan 2024-09-30T16:01:18+00:00

Last Scan

Scanned2024-09-23T16:01:18+00:00
URL https://cntravellerme.com/robots.txt
Redirect https://www.cntravellerme.com/robots.txt
Redirect Domain www.cntravellerme.com
Redirect Base cntravellerme.com
Domain IPs 151.101.130.137, 151.101.194.137, 151.101.2.137, 151.101.66.137
Redirect IPs 13.35.18.57, 13.35.18.58, 13.35.18.63, 13.35.18.67, 2600:9000:20c7:1400:18:86ba:fe80:93a1, 2600:9000:20c7:5600:18:86ba:fe80:93a1, 2600:9000:20c7:7200:18:86ba:fe80:93a1, 2600:9000:20c7:8600:18:86ba:fe80:93a1, 2600:9000:20c7:9e00:18:86ba:fe80:93a1, 2600:9000:20c7:b600:18:86ba:fe80:93a1, 2600:9000:20c7:c200:18:86ba:fe80:93a1, 2600:9000:20c7:d600:18:86ba:fe80:93a1
Response IP 13.35.18.67
Found Yes
Hash 04491cb3262332f75ce06d328a557a1d391911c00f446c0fdf8beee11ca579fe
SimHash 648ad941ce33

Groups

*

Rule Path
Disallow /preview/
Disallow /search
Disallow /*%7B%7Bsection%7D%7D
Disallow /*%7B%7Bsection%7D%7D
Disallow /product/
Disallow /auth/*
Disallow /account/*
Disallow /*?redirectURL=
Disallow /user-context?referrer
Disallow /services.min.js?_=

google-inspectiontool

Rule Path
Allow /

ccbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

youbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.cntravellerme.com/sitemap.xml
sitemap https://www.cntravellerme.com/categories-sitemap.xml
sitemap https://www.cntravellerme.com/contributors-sitemap.xml
sitemap https://www.cntravellerme.com/bundles-sitemap.xml
sitemap https://www.cntravellerme.com/feed/google-latest-news/sitemap-google-news
sitemap https://www.cntravellerme.com/feed/rss