agetimes.co.uk
robots.txt
Robots Exclusion Standard data for agetimes.co.uk
Resource Scan
Scan Details
| Site Domain | agetimes.co.uk |
| Base Domain | agetimes.co.uk |
| Scan Status | Ok |
| Last Scan | 2024-11-09T17:08:13+00:00 |
| Next Scan | 2024-11-16T17:08:13+00:00 |
Last Scan
| Scanned | 2024-11-09T17:08:13+00:00 |
| URL | https://agetimes.co.uk/robots.txt |
| Redirect | https://www.agetimes.co.uk/robots.txt |
| Redirect Domain | www.agetimes.co.uk |
| Redirect Base | agetimes.co.uk |
| Domain IPs | 104.21.83.124, 172.67.175.215, 2606:4700:3031::ac43:afd7, 2606:4700:3036::6815:537c |
| Redirect IPs | 104.21.83.124, 172.67.175.215, 2606:4700:3031::ac43:afd7, 2606:4700:3036::6815:537c |
| Response IP | 172.67.175.215 |
| Found | Yes |
| Hash | 1ed8bc5b2741a428dde80035e8f3239a5f973a7de19f1c11cf2c1e006f6a20ef |
| SimHash | 4c04d8218fc3 |
Groups
datocmssearchbot
| Rule | Path |
|---|---|
| Disallow | /**/*-agtestag |
| Disallow | */amp |
| Disallow | /amp |
| Disallow | /authors/* |
| Disallow | /search |
| Disallow | /landing |
| Disallow | /ad-test-a |
| Disallow | /ad-test-b |
*
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /**/*-agtestag |
| Disallow | /landing |
| Disallow | /ad-test-a |
| Disallow | /ad-test-b |
| Disallow | /**/page |
| Disallow | /*page-data.json |
| Disallow | /intermediate |
| Disallow | /search |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.agetimes.co.uk/sitemap-0.xml |
Warnings
- `host` is not a known field.