eurocheapo.com
robots.txt

Robots Exclusion Standard data for eurocheapo.com

Resource Scan

Scan Details

Site Domain eurocheapo.com
Base Domain eurocheapo.com
Scan Status Ok
Last Scan2024-05-28T23:50:00+00:00
Next Scan 2024-06-04T23:50:00+00:00

Last Scan

Scanned2024-05-28T23:50:00+00:00
URL https://eurocheapo.com/robots.txt
Redirect https://www.eurocheapo.com/robots.txt
Redirect Domain www.eurocheapo.com
Redirect Base eurocheapo.com
Domain IPs 172.66.40.144, 172.66.43.112
Redirect IPs 172.66.40.144, 172.66.43.112
Response IP 172.66.43.112
Found Yes
Hash a734af0429bc19727b32760682efa59983583af40d425c1f589b36ec3f6e3c01
SimHash 584c6550cdf1

Groups

*

Rule Path
Disallow /cgi-bin/
Disallow /continue/
Disallow /bounce/
Disallow /flight/
Disallow /flights/routesearch/

expo9

Rule Path
Disallow /

owlin

Rule Path
Disallow /

owlin bot

Rule Path
Disallow /

owlin bot v3

Rule Path
Disallow /

nutch

Rule Path
Disallow /

getintentcrawler

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

proximic

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

easouspider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

wesee

Rule Path
Disallow /

umbot-ln

Rule Path
Disallow /

umbot-ln/1.0

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

blekkobot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

yandex

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

Other Records

Field Value
sitemap http://www.eurocheapo.com/sitemaps/ec-airlines.xml
sitemap http://www.eurocheapo.com/sitemaps/ec-airports.xml
sitemap http://www.eurocheapo.com/sitemaps/ec-articles.xml
sitemap http://www.eurocheapo.com/sitemaps/ec-cities.xml
sitemap http://www.eurocheapo.com/sitemaps/ec-hoods.xml
sitemap http://www.eurocheapo.com/sitemaps/ec-price.xml
sitemap http://www.eurocheapo.com/blog/post-sitemap1.xml
sitemap http://www.eurocheapo.com/blog/post-sitemap2.xml
sitemap http://www.eurocheapo.com/blog/post-sitemap3.xml
sitemap http://www.eurocheapo.com/blog/sitemap_index.xml

Comments

  • various robots that get the Cheapo Banhammer
  • feel proud that you've got a listing in here, it means you're a 429 champion
  • rate limited robots below

Warnings

  • 2 invalid lines.