thewreckingcrew.eu
robots.txt
Robots Exclusion Standard data for thewreckingcrew.eu
Resource Scan
Scan Details
Site Domain | thewreckingcrew.eu |
Base Domain | thewreckingcrew.eu |
Scan Status | Failed |
Failure Stage | Fetching resource. |
Failure Reason | Server returned a server error. |
Last Scan | 2024-07-09T23:30:22+00:00 |
Next Scan | 2024-10-07T23:30:22+00:00 |
Last Successful Scan
Scanned | 2024-02-18T23:28:32+00:00 |
URL | http://thewreckingcrew.eu/robots.txt |
Redirect | http://www.thewreckingcrew.eu/robots.txt |
Redirect Domain | www.thewreckingcrew.eu |
Redirect Base | thewreckingcrew.eu |
Domain IPs | 5.10.98.108 |
Redirect IPs | 5.10.98.108 |
Response IP | 5.10.98.108 |
Found | Yes |
Hash | 0d636d479f1828c0a1bc03e61979f4e2dd076b1822109192756ae41b09222409 |
SimHash | 739ff25acd62 |
Groups
*
Rule | Path |
---|---|
Disallow | /admin |
Disallow | /?action |
Other Records
Field | Value |
---|---|
crawl-delay | 60 |
Other Records
Field | Value |
---|---|
sitemap | http://www.thewreckingcrew.eu/sitemap.xml |