all4.com
robots.txt
Robots Exclusion Standard data for all4.com
Resource Scan
Scan Details
| Site Domain | all4.com |
| Base Domain | all4.com |
| Scan Status | Ok |
| Last Scan | 2024-06-11T01:26:20+00:00 |
| Next Scan | 2024-06-18T01:26:20+00:00 |
Last Scan
| Scanned | 2024-06-11T01:26:20+00:00 |
| URL | https://all4.com/robots.txt |
| Redirect | https://www.channel4.com/robots.txt |
| Redirect Domain | www.channel4.com |
| Redirect Base | channel4.com |
| Domain IPs | 54.77.183.181, 99.80.55.79 |
| Redirect IPs | 23.53.217.24 |
| Response IP | 23.54.57.184 |
| Found | Yes |
| Hash | e8d33369a253562578e768334518d47bf9893a1088c21a24c1f961035d113094 |
| SimHash | e9015c70c333 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /news/?* |
| Disallow | /news/*/?* |
| Disallow | /press/unregistered-image-search?* |
| Disallow | /press/content-search?* |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.channel4.com/news/sitemap.xml |
| sitemap | https://www.channel4.com/sitemap.xml |
Warnings
- 1 invalid line.