travel.india.com
robots.txt

Robots Exclusion Standard data for travel.india.com

Resource Scan

Scan Details

Site Domain travel.india.com
Base Domain india.com
Scan Status Ok
Last Scan2025-03-03T08:08:04+00:00
Next Scan 2025-04-02T08:08:04+00:00

Last Scan

Scanned2025-03-03T08:08:04+00:00
URL https://travel.india.com/robots.txt
Domain IPs 104.88.71.11, 104.88.71.113, 2600:1413:b000:6::17d5:2bc8, 2600:1413:b000:6::17d5:2bd0
Response IP 23.54.118.45
Found Yes
Hash 623747d7b7c14f22693bb7551d36a6d8b5156cde01bae2dcd3c0c088f6aba8dc
SimHash 311d9810cd33

Groups

*

Rule Path
Allow /

petalbot

Rule Path
Disallow /

urxbot/*

Rule Path
Disallow

urx-api/*

Rule Path
Disallow

baiduspider

Rule Path
Disallow /

yandex

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

semrush

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

boardreader

Rule Path
Disallow /

boardreader

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

trendiction

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

trendiction.com

Rule Path
Disallow /

trendiction.de

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

Other Records

Field Value
sitemap https://travel.india.com/travel-googlenews.xml
sitemap https://travel.india.com/web-stories/travel-googlenews.xml
sitemap https://travel.india.com/sitemap-section.xml
sitemap https://travel.india.com/travel-complete-sitemap.xml
sitemap https://travel.india.com/travel-hindi-googlenews.xml
sitemap https://travel.india.com/hindi-news/web-stories/travel-googlenews.xml
sitemap https://travel.india.com/hindi-news/sitemap-section.xml

Comments

  • Baiduspider
  • Yandex