automotioncomponents.co.uk
robots.txt
Robots Exclusion Standard data for automotioncomponents.co.uk
Resource Scan
Scan Details
Site Domain | automotioncomponents.co.uk |
Base Domain | automotioncomponents.co.uk |
Scan Status | Ok |
Last Scan | 2024-09-17T04:41:20+00:00 |
Next Scan | 2024-10-17T04:41:20+00:00 |
Last Scan
Scanned | 2024-09-17T04:41:20+00:00 |
URL | https://automotioncomponents.co.uk/robots.txt |
Redirect | https://www.automotioncomponents.co.uk/robots.txt |
Redirect Domain | www.automotioncomponents.co.uk |
Redirect Base | automotioncomponents.co.uk |
Domain IPs | 20.50.64.20 |
Redirect IPs | 96.17.96.11, 96.17.96.27 |
Response IP | 104.88.70.137 |
Found | Yes |
Hash | e06374fba9437f0914d3c4be5c6584b5cd818cabe281e9a902695eb45cb7a371 |
SimHash | ac14da74c393 |
Groups
*
Rule | Path |
---|---|
Disallow | /admin/ |
Disallow | /js/ |
Other Records
Field | Value |
---|---|
crawl-delay | 5 |