Log In Sign Up

billyrider.de
robots.txt

Robots Exclusion Standard data for billyrider.de

Resource Scan

Scan Details

Site Domain billyrider.de
Base Domain billyrider.de
Scan Status Ok
Last Scan2024-06-28T06:45:49+00:00
Next Scan 2024-07-05T06:45:49+00:00

Last Scan

Scanned2024-06-28T06:45:49+00:00
URL https://billyrider.de/robots.txt
Domain IPs 5.9.21.39
Response IP 5.9.21.39
Found Yes
Hash 4f9fb8a2375ce93d0463d325784ec49ffa52301e84e4179ddeed92574af03118
SimHash da5e58d0c398

Groups

*

Rule Path
Disallow /da/
Disallow /revi
Disallow /search?*
Disallow /login?*
Disallow /rgstr?*
Disallow /out?*

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

amazonbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

applebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

awariorssbot
awariosmartbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

blexbot

Rule Path
Disallow /

claudebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

dataforseobot

Rule Path
Disallow /

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

imagesiftbot

Rule Path
Disallow /

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

geedobot

Rule Path
Disallow /

geedoproductsearch

Rule Path
Disallow /

petalbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

seznambot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

twitterbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

yandexaccessibilitybot
yandexadditional
yandexadditionalbot
yandexadnet
yandexblogs
yandexbot
yandexcalendar
yandexdirect
yandexdirectdyn
yandexfavicons
yadirectfetcher
yandexfordomain
yandeximages
yandeximageresizer
yandexmobilebot
yandexmarket
yandexmedia
yandexmetrika
yandexmobilescreenshotbot
yandexnews
yandexontodb
yandexontodbapi
yandexpagechecker
yandexpartner
yandexrca
yandexrenderresourcesbot
yandexscreenshotbot
yandexsearchshop
yandexsitelinks
yandexspravbot
yandextracker
yandexturbo
yandexuserproxy
yandexvertis
yandexverticals
yandexvideo
yandexvideoparser
yandexwebmaster

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10
Back to top