cs.ox.ac.uk
robots.txt
Robots Exclusion Standard data for cs.ox.ac.uk
Resource Scan
Scan Details
| Site Domain | cs.ox.ac.uk |
| Base Domain | ox.ac.uk |
| Scan Status | Ok |
| Last Scan | 2025-05-26T10:00:48+00:00 |
| Next Scan | 2025-06-25T10:00:48+00:00 |
Last Scan
| Scanned | 2025-05-26T10:00:48+00:00 |
| URL | https://cs.ox.ac.uk/robots.txt |
| Redirect | https://www.cs.ox.ac.uk/robots.txt |
| Redirect Domain | www.cs.ox.ac.uk |
| Redirect Base | ox.ac.uk |
| Domain IPs | 129.67.151.1 |
| Redirect IPs | 129.67.151.1 |
| Response IP | 129.67.151.1 |
| Found | Yes |
| Hash | 8077af1d53f4e1e9fbbdb06fcc6cc13ec1bdc9d797d7881a8c582231d970b85d |
| SimHash | 1a129d49dff2 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /blogs/bncod2013/ |
| Disallow | /teaching/material |
| Disallow | /teaching/material/ |
| Disallow | /teaching/internal |
| Disallow | /teaching/internal/ |
| Disallow | /internal |
| Disallow | /internal/ |
| Disallow | /personal/teaching/materials08-09/ |
| Disallow | /webalizer/ |
| Disallow | /files/268/ |
| Disallow | /minerva/ |
| Disallow | /files/1217/Handbook%202008%20v3.pdf |
| Disallow | /files/3265/Handbook%202010.pdf |
| Disallow | /files/391/info07msc.pdf |
| Disallow | /files/3669/handbook-10-11.pdf |
| Disallow | /degrees/documents/prs10.pdf |
| Disallow | /files/4063/costWMSO_MFCS2011.pdf |
| Disallow | /people/maneesh.khattri/ |
| Disallow | /events/*.jsp |
| Disallow | /dynamicfeed/ |
| Disallow | /booster/ |
| Disallow | /signin/ |
Other Records
| Field | Value | Comment |
|---|---|---|
| crawl-delay | 1 | Might slow down the crawlers - ignored by Google |
Other Records
| Field | Value |
|---|---|
| sitemap | http://www.softeng.ox.ac.uk/sitemap.xml |
Comments