economybookings.com
robots.txt

Robots Exclusion Standard data for economybookings.com

Resource Scan

Scan Details

Site Domain economybookings.com
Base Domain economybookings.com
Scan Status Ok
Last Scan2024-06-01T05:18:30+00:00
Next Scan 2024-06-15T05:18:30+00:00

Last Scan

Scanned2024-06-01T05:18:30+00:00
URL https://economybookings.com/robots.txt
Redirect https://www.economybookings.com:443/robots.txt
Redirect Domain www.economybookings.com
Redirect Base economybookings.com
Domain IPs 52.214.166.69, 63.33.31.129
Redirect IPs 104.22.12.26, 104.22.13.26, 172.67.15.24, 2606:4700:10::6816:c1a, 2606:4700:10::6816:d1a, 2606:4700:10::ac43:f18
Response IP 104.22.13.26
Found Yes
Hash 4fc8f038b63199050f037d3995d69a166e1a5a3e0fa1eba450938791871b0572
SimHash 0105c870e372

Groups

turnitinbot

Rule Path
Disallow /

npbot-1/2.0

Rule Path
Disallow /

npbot

Rule Path
Disallow /

psbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

*

Rule Path
Disallow /invoice
Disallow /voucher/*
Disallow /*/voucher/*
Disallow /invoice/*
Disallow /*/invoice/*
Disallow /*/cars/results?personal_cab*
Disallow /cars/results?personal_cab*
Disallow /cars/results/
Disallow /cars/
Disallow /*/cars/
Disallow /*?idpickval
Disallow /*?idpick
Disallow /*/?idpickval
Disallow /*/?idpick
Disallow /addExtra/
Disallow /*/addExtra/
Disallow /voucher/
Disallow /?op=reg
Disallow /*/?op=reg
Disallow /fb
Disallow /*/fb

Other Records

Field Value
crawl-delay 10

adsbot-google
adsbot-google-mobile

Rule Path
Allow /*?idpickval
Allow /*?idpick
Allow /*/?idpickval
Allow /*/?idpick

baiduspider-ads

Rule Path
Allow /*

baiduspider

Rule Path
Allow /zh/
Disallow /de/
Disallow /ee/
Disallow /es/
Disallow /fi/
Disallow /fr/
Disallow /it/
Disallow /lt/
Disallow /lv/
Disallow /nl/
Disallow /pl/
Disallow /pt/
Disallow /ru/
Disallow /sv/
Disallow /bg/
Disallow /sv/

yahoo!-adcrawler

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 0.5

adidxbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 3

mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com)

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 3

Other Records

Field Value
sitemap https://www.economybookings.com/sitemaps/sitemapindex.xml

Comments

  • 3rd party sites
  • Paths (clean URLs)
  • Paths

Warnings

  • `host` is not a known field.