city.ac.uk
robots.txt
Robots Exclusion Standard data for city.ac.uk
Resource Scan
Scan Details
| Site Domain | city.ac.uk |
| Base Domain | city.ac.uk |
| Scan Status | Ok |
| Last Scan | 2024-09-20T04:22:57+00:00 |
| Next Scan | 2024-10-20T04:22:57+00:00 |
Last Scan
| Scanned | 2024-09-20T04:22:57+00:00 |
| URL | https://city.ac.uk/robots.txt |
| Redirect | https://www.city.ac.uk/robots.txt |
| Redirect Domain | www.city.ac.uk |
| Redirect Base | city.ac.uk |
| Domain IPs | 43.245.41.27 |
| Redirect IPs | 104.18.32.26, 172.64.155.230 |
| Response IP | 104.18.32.26 |
| Found | Yes |
| Hash | ec7ed4cabea18218f33a9766694dd25878ae8ca7831a44d108365558dae9186b |
| SimHash | 0048161604d7 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /visit/feeds |
| Disallow | /web-services |
| Disallow | /apis |
| Disallow | /api |
| Disallow | /staging |
| Disallow | /tools |
| Disallow | /documentation |
| Disallow | /forms |
| Disallow | /applicant |
| Disallow | /_media |
| Disallow | /news/city-statements |
| Disallow | /__old_design |
| Disallow | /_old_design |
| Disallow | /*?SQ_VARIATION* |
| Disallow | /home |
| Disallow | /home-archived |
| Disallow | /*?dev=* |
| Disallow | /*?rel=* |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.city.ac.uk/sitemap.xml |