deutschroutenplaner.de
robots.txt

Robots Exclusion Standard data for deutschroutenplaner.de

Resource Scan

Scan Details

Site Domain deutschroutenplaner.de
Base Domain deutschroutenplaner.de
Scan Status Ok
Last Scan2024-05-25T16:26:01+00:00
Next Scan 2024-06-24T16:26:01+00:00

Last Scan

Scanned2024-05-25T16:26:01+00:00
URL https://deutschroutenplaner.de/robots.txt
Redirect https://www.deutschroutenplaner.de/robots.txt
Redirect Domain www.deutschroutenplaner.de
Redirect Base deutschroutenplaner.de
Domain IPs 104.21.69.57, 172.67.204.235, 2606:4700:3037::6815:4539, 2606:4700:3037::ac43:cceb
Redirect IPs 104.21.69.57, 172.67.204.235, 2606:4700:3037::6815:4539, 2606:4700:3037::ac43:cceb
Response IP 104.21.69.57
Found Yes
Hash f67db14b654199024f294106f15d5ab5495ee53a33d1dc16af8a3d078a2e2f40
SimHash 564543d0c7c8

Groups

baiduspider

Rule Path
Disallow /

baiduspider-video

Rule Path
Disallow /

baiduspider-image

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

megaindex.ru/2.0

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

psbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

npbot-1/2.0

Rule Path
Disallow /

npbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

domainstatsbot

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

coccocbot-web

Rule Path
Disallow /

linespider

Rule Path
Disallow /

infotigerbot

Rule Path
Disallow /

adsbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

zoombot

Rule Path
Disallow /

admantx

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

neevabot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

applebot

Rule Path
Disallow /

amazon-kendra-web-crawler-*

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

*

Rule Path
Disallow /pub/
Disallow /inc/
Disallow /js/
Disallow /action/
Disallow /policies/