travelai.com
robots.txt

Robots Exclusion Standard data for travelai.com

Resource Scan

Scan Details

Site Domain travelai.com
Base Domain travelai.com
Scan Status Ok
Last Scan2024-11-06T20:28:13+00:00
Next Scan 2024-12-06T20:28:13+00:00

Last Scan

Scanned2024-11-06T20:28:13+00:00
URL https://travelai.com/robots.txt
Redirect https://www.travelai.com/robots.txt
Redirect Domain www.travelai.com
Redirect Base travelai.com
Domain IPs 13.35.210.15, 13.35.210.19, 13.35.210.25, 13.35.210.68
Redirect IPs 13.33.88.51, 13.33.88.55, 13.33.88.61, 13.33.88.9
Response IP 13.33.88.61
Found Yes
Hash 77ddad495e221e2da2a3b534fac700d7c0e9616e93efbe8619fb09cff3f87043
SimHash 7e3a4277fdb3

Groups

*

Rule Path
Disallow /api/
Disallow /creator-program

bubing
genieo
grapeshot
jamesbot
ltx71 - (http://ltx71.com/)
mj12bot
seznambot
dotbot
ccbot
tineye-bot
pubmatic crawler bot
mappydata
jooblebot
grapeshot
domainstatsbot
pubmaticcrawlerbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://cdn.travelai.com/site/sitemap.xml
sitemap https://www.travelai.com/resources/sitemap.xml