dundee.ac.uk
robots.txt

Robots Exclusion Standard data for dundee.ac.uk

Resource Scan

Scan Details

Site Domain dundee.ac.uk
Base Domain dundee.ac.uk
Scan Status Ok
Last Scan2024-05-31T08:25:46+00:00
Next Scan 2024-06-30T08:25:46+00:00

Last Scan

Scanned2024-05-31T08:25:46+00:00
URL https://dundee.ac.uk/robots.txt
Redirect https://www.dundee.ac.uk/robots.txt
Redirect Domain www.dundee.ac.uk
Redirect Base dundee.ac.uk
Domain IPs 134.36.2.74
Redirect IPs 134.36.2.74
Response IP 134.36.2.74
Found Yes
Hash 29b552dd1fb66d7330438d1f5565c5ccc420c1dc36c1a4d9a8ac086f21513087
SimHash 89129f984bf2

Groups

*

Rule Path
Disallow /virtualhosts/
Disallow /media/
Disallow /xml/
Disallow /_lib/
Disallow /home/
Disallow /study/pg/education/esw-download/
Disallow /study/pg/psychological-therapy-primary-care/downloads/
Disallow /study/pg/education/downloads/
Disallow /search
Disallow /landing-page-paragraphs-qa

*

Rule Path
Allow /media/dundeewebsite/themes/
Allow /media/dundeewebsite/src/css/
Allow /media/dundeewebsite/src/img/layout/
Allow /media/dundeewebsite/homepage/carousel/
Allow /media/dundeewebsite/vendors/
Allow /media/dundeewebsite/main/news/

*

Rule Path
Disallow /governance/academic-calendar/

funnelback

Rule Path
Allow /

sitecheck-sitecrawl by siteimprove.com

Rule Path
Allow /

linkcheck by siteimprove.com

Rule Path
Allow /

trendkite-akashic-crawler

Rule Path
Disallow /

Comments

  • robots.txt for https://www.dundee.ac.uk/
  • Contact: web-administrator@dundee.ac.uk
  • Last modified 04 January 2016
  • Block trendkite-akashic-crawler