fußballdaten.de
robots.txt

Robots Exclusion Standard data for fußballdaten.de

Resource Scan

Scan Details

Site Domain fußballdaten.de
Base Domain fußballdaten.de
Scan Status Ok
Last Scan2024-11-09T01:16:24+00:00
Next Scan 2024-11-16T01:16:24+00:00

Last Scan

Scanned2024-11-09T01:16:24+00:00
URL http://fußballdaten.de/robots.txt
Redirect https://www.fussballdaten.de/robots.txt
Redirect Domain www.fussballdaten.de
Redirect Base fussballdaten.de
Domain IPs 5.9.13.40
Redirect IPs 5.9.13.40
Response IP 5.9.13.40
Found Yes
Hash e47bdf939afa9136669d9df5f1bc1c671327bf9424d24d0f9a9a1560e8ffc408
SimHash 7318d871c417

Groups

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

wget

Rule Path
Disallow /

*

Rule Path
Disallow /*json-*
Disallow /wettbewerb/tabelle-startseite/
Disallow /news/monat-ajax/
Allow /

Other Records

Field Value
sitemap https://www.fussballdaten.de/sitemap.xml