sf-fan.de
robots.txt

Robots Exclusion Standard data for sf-fan.de

Resource Scan

Scan Details

Site Domain sf-fan.de
Base Domain sf-fan.de
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2025-07-11T15:18:54+00:00
Next Scan 2025-09-09T15:18:54+00:00

Last Successful Scan

Scanned2025-05-13T15:17:20+00:00
URL https://sf-fan.de/robots.txt
Domain IPs 2a01:238:434a:f900:7570:3172:9954:cf2b, 85.214.162.144
Response IP 85.214.162.144
Found Yes
Hash 128c1aaee249a67bc2a3e9db88516b3908c1c75a8851e444a3a1967160ea083e
SimHash ab7ad4d06870

Groups

*

Rule Path
Disallow /administrator/
Disallow /cache/
Disallow /components/
Disallow /editor/
Disallow /help/
Disallow /includes/
Disallow /language/
Disallow /mambots/
Disallow /media/
Disallow /modules/
Disallow /installation/

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 200

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 100

teoma

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 300

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 200

francis

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

seekbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 300

Warnings

  • 2 invalid lines.