cosmopolitan.co.uk
robots.txt
Robots Exclusion Standard data for cosmopolitan.co.uk
Resource Scan
Scan Details
| Site Domain | cosmopolitan.co.uk |
| Base Domain | cosmopolitan.co.uk |
| Scan Status | Ok |
| Last Scan | 2024-11-09T10:22:52+00:00 |
| Next Scan | 2024-11-16T10:22:52+00:00 |
Last Scan
| Scanned | 2024-11-09T10:22:52+00:00 |
| URL | https://www.cosmopolitan.co.uk/robots.txt |
| Domain IPs | 151.101.0.200, 151.101.128.200, 151.101.192.200, 151.101.64.200 |
| Response IP | 199.232.44.200 |
| Found | Yes |
| Hash | 6d91b8069474f7bbae481d591d87b8ea3bbd8e042072b72581c87b15a5d776ff |
| SimHash | 63385c4d4410 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /api/ |
| Disallow | /oauth/ |
| Disallow | /search/ |
| Disallow | /search-fetch/ |
| Disallow | /ajax/ |
| Disallow | /landing-feed/ |
| Disallow | /transporter/ |
| Disallow | /chatroom/ |
| Disallow | /print-this/ |
| Disallow | /hmiuk-cosmopolitan/ |
Other Records
| Field | Value |
|---|---|
| sitemap | http://www.cosmopolitan.co.uk/sitemap_index.xml |