Log In Sign Up

lifelong-learning.ox.ac.uk
robots.txt

Robots Exclusion Standard data for lifelong-learning.ox.ac.uk

Resource Scan

Scan Details

Site Domain lifelong-learning.ox.ac.uk
Base Domain ox.ac.uk
Scan Status Ok
Last Scan2025-09-10T02:01:51+00:00
Next Scan 2025-10-10T02:01:51+00:00

Last Scan

Scanned2025-09-10T02:01:51+00:00
URL https://lifelong-learning.ox.ac.uk/robots.txt
Domain IPs 129.67.165.148
Response IP 129.67.165.148
Found Yes
Hash 3d7e2c4f9ace200c4e54d52d782fab69687f1be28ecec6deb4756d28c60c993f
SimHash 406be859e7d1

Groups

*

Rule Path Comment
Disallow *.pdf Block pdf files. Non-standard but works for major search engines.
Disallow /basket/add-module*$ -
Disallow /error$ -
Disallow /events/log-payment/*$ Block any log payment events functions to be crawled.
Disallow /events/register/*$ -
Disallow /about-us -
Disallow /about/continuing-education-library-service-during-closure -
Disallow /mnt/archive/*$ -
Disallow /mnt/attachments/*$ -
Disallow /mnt/audio/*$ -
Disallow /mnt/forms/*$ -

googlebot-video

Rule Path
Disallow /mnt/video/*$
Disallow /basket
Disallow /myaccount
Back to top

Comments

  • Breaking down what is blocked by the Googlebot in the mnt folder
Back to top

Warnings

  • 1 invalid line.
Back to top