starwars-union.de
robots.txt

Robots Exclusion Standard data for starwars-union.de

Resource Scan

Scan Details

Site Domain starwars-union.de
Base Domain starwars-union.de
Scan Status Ok
Last Scan2024-09-22T22:14:01+00:00
Next Scan 2024-09-29T22:14:01+00:00

Last Scan

Scanned2024-09-22T22:14:01+00:00
URL https://starwars-union.de/robots.txt
Redirect https://www.starwars-union.de/robots.txt
Redirect Domain www.starwars-union.de
Redirect Base starwars-union.de
Domain IPs 217.160.0.64
Redirect IPs 2001:8d8:100f:f000::217, 217.160.0.64
Response IP 217.160.0.64
Found Yes
Hash 94c11f9ce49012525b9fdc94b34f333ef617a6f51419253c47246b3d1be33fe6
SimHash 6814fcd080b1

Groups

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 50

*

Rule Path
Disallow /admin/
Disallow /mods/

Other Records

Field Value
sitemap https://www.starwars-union.de/sitemap.php

Warnings

  • 2 invalid lines.