reading.gov.uk
robots.txt
Robots Exclusion Standard data for reading.gov.uk
Resource Scan
Scan Details
| Site Domain | reading.gov.uk |
| Base Domain | reading.gov.uk |
| Scan Status | Ok |
| Last Scan | 2024-11-03T20:39:17+00:00 |
| Next Scan | 2024-11-17T20:39:17+00:00 |
Last Scan
| Scanned | 2024-11-03T20:39:17+00:00 |
| URL | https://reading.gov.uk/robots.txt |
| Redirect | https://www.reading.gov.uk/robots.txt |
| Redirect Domain | www.reading.gov.uk |
| Redirect Base | reading.gov.uk |
| Domain IPs | 35.178.153.235, 52.56.126.239 |
| Redirect IPs | 13.40.232.110, 18.135.252.146 |
| Response IP | 18.135.252.146 |
| Found | Yes |
| Hash | b582996fc247573f4b060c16ea9b981134b2ec0b20ac914057e4b15abd8a8cf3 |
| SimHash | 411c5c442693 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /search/ |
| Disallow | /leisure/things-to-do/water-fest/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.reading.gov.uk/sitemap.xml |
Comments