chrono24.com
robots.txt

Robots Exclusion Standard data for chrono24.com

Resource Scan

Scan Details

Site Domain chrono24.com
Base Domain chrono24.com
Scan Status Ok
Last Scan2024-05-21T18:05:15+00:00
Next Scan 2024-06-20T18:05:15+00:00

Last Scan

Scanned2024-05-21T18:05:15+00:00
URL https://chrono24.com/robots.txt
Redirect https://www.chrono24.com/robots.txt
Redirect Domain www.chrono24.com
Redirect Base chrono24.com
Domain IPs 104.17.217.13, 104.17.218.13
Redirect IPs 104.17.217.13, 104.17.218.13
Response IP 104.17.218.13
Found Yes
Hash dec8f7c2431296726bf57b76483f231856a3b538ffdd9936d448314553bfe882
SimHash 524b1fe91740

Groups

seekbot

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

pompos

Rule Path
Disallow /

solomono

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

linguee

Rule Path
Disallow /

lwnutch

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

riddlerbot

Rule Path
Disallow /

discoverybot

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

endeca webcrawler

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

httrack website copier

Rule Path
Disallow /

sentibot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

contextad bot

Rule Path
Disallow /

umbot

Rule Path
Disallow /

rubrikkbot

Rule Path
Disallow /

admantx

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

seodiver

Rule Path
Disallow /

plukkie

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

wesee

Rule Path
Disallow /

psbot

Rule Path
Disallow /

bubing

Rule Path
Disallow /

voltron

Rule Path
Disallow /

proximic

Rule Path
Disallow /

riddler

Rule Path
Disallow /

bnsbot

Rule Path
Disallow /

mnogosearch

Rule Path
Disallow /

um-fc

Rule Path
Disallow /

domaincrawler

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

domaincrawler

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

bdcbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

searchmetricsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 0.5

petalbot

Rule Path
Disallow */dealer/checkout-rating.htm*

googlebot

Rule Path
Disallow /lib/ads/*
Disallow /lib/generated/js/*
Disallow */others/ajax_picturemover.htm*
Disallow */others/popup_*
Disallow */others/iframe_*
Disallow */others/banner_link.htm*
Disallow */others/buynow_link.htm*
Disallow */others/clicktrack.htm*
Disallow */others/adIframe*
Disallow */url/military-watches*
Disallow *url%2*military-watches*
Disallow */search/suggest.htm*
Disallow */search/similar-watches.htm*
Disallow */search/detail.htm*%26notes*
Disallow */search/detail.htm*%26originalNotes*

*

Rule Path
Disallow /lib/ads/*
Disallow /lib/generated/js/*
Disallow */others/ajax_picturemover.htm*
Disallow */others/popup_*
Disallow */others/iframe_*
Disallow */others/banner_link.htm*
Disallow */others/buynow_link.htm*
Disallow */others/clicktrack.htm*
Disallow */others/adIframe*
Disallow */search/similar-watches.htm*
Disallow */search/modelrating.htm*
Disallow */others/quickreply.htm*
Disallow */user/activate.htm*

Comments

  • robots.txt for http://www.chrono24.com