scottishhellfighters.clanwebsite.com
robots.txt
Robots Exclusion Standard data for scottishhellfighters.clanwebsite.com
Resource Scan
Scan Details
Site Domain | scottishhellfighters.clanwebsite.com |
Base Domain | clanwebsite.com |
Scan Status | Failed |
Failure Stage | Fetching resource. |
Failure Reason | Server returned a server error. |
Last Scan | 2024-09-27T14:02:41+00:00 |
Next Scan | 2024-12-26T14:02:41+00:00 |
Last Successful Scan
Scanned | 2024-02-08T12:58:48+00:00 |
URL | http://scottishhellfighters.clanwebsite.com/robots.txt |
Domain IPs | 5.10.98.111 |
Response IP | 5.10.98.111 |
Found | Yes |
Hash | 43f26f53fc55a849c97abb4f51be027398839d024c6e26d22825fef9bea16778 |
SimHash | 73b7f27acd62 |
Groups
*
Rule | Path |
---|---|
Disallow | /admin |
Disallow | /?action |
Other Records
Field | Value |
---|---|
crawl-delay | 60 |
Other Records
Field | Value |
---|---|
sitemap | http://scottishhellfighters.clanwebsite.com/sitemap.xml |