tipps-archiv.de
robots.txt

Robots Exclusion Standard data for tipps-archiv.de

Resource Scan

Scan Details

Site Domain tipps-archiv.de
Base Domain tipps-archiv.de
Scan Status Ok
Last Scan2024-06-07T09:55:19+00:00
Next Scan 2024-06-14T09:55:19+00:00

Last Scan

Scanned2024-06-07T09:55:19+00:00
URL https://tipps-archiv.de/robots.txt
Domain IPs 5.75.191.22
Response IP 5.75.191.22
Found Yes
Hash 1cb7ae8e20d6d41a2103f1bb2c203b3601bcb6b4ac1e119784e7f9c7c0798960
SimHash 693fd24157c0

Groups

awariorssbot
awariosmartbot

Rule Path
Disallow /tag/

awariorssbot
awariosmartbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

bingbot

Rule Path
Disallow /tag/

Other Records

Field Value
crawl-delay 300

googlebot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 15

semrushbot

Rule Path
Disallow /tag/

*

Rule Path
Disallow /cgi-bin/
Disallow /scripts/
Disallow /tag/

Other Records

Field Value
crawl-delay 360