lut.ac.uk
robots.txt

Robots Exclusion Standard data for lut.ac.uk

Resource Scan

Scan Details

Site Domain lut.ac.uk
Base Domain lut.ac.uk
Scan Status Ok
Last Scan2025-09-16T17:57:46+00:00
Next Scan 2025-10-16T17:57:46+00:00

Last Scan

Scanned2025-09-16T17:57:46+00:00
URL https://lut.ac.uk/robots.txt
Redirect https://www.lboro.ac.uk/robots.txt
Redirect Domain www.lboro.ac.uk
Redirect Base lboro.ac.uk
Domain IPs 75.2.70.143, 76.223.116.192
Redirect IPs 13.33.45.103, 13.33.45.23, 13.33.45.71, 13.33.45.98, 2600:9000:229f:1200:11:53c2:9bc0:93a1, 2600:9000:229f:5800:11:53c2:9bc0:93a1, 2600:9000:229f:6000:11:53c2:9bc0:93a1, 2600:9000:229f:7400:11:53c2:9bc0:93a1, 2600:9000:229f:f000:11:53c2:9bc0:93a1, 2600:9000:229f:f200:11:53c2:9bc0:93a1, 2600:9000:229f:f800:11:53c2:9bc0:93a1, 2600:9000:229f:fc00:11:53c2:9bc0:93a1
Response IP 13.33.45.98
Found Yes
Hash 9bb3dc07ba7b8b84695d21be12fc390449765c79f0a77090c4035cfe139cb13e
SimHash 7931cb2f4fe5

Groups

*

Product Comment
* directed to all spiders, not just Scooter
Rule Path
Disallow /departments/ac/
Disallow /cgi-bin/
Disallow /usage/
Disallow /Excite/
Disallow /dir/
Disallow /news/notices/
Disallow /computing/hallnet-test/
Disallow /mediaservices/ex/
Disallow /designandprint/ex/
Disallow /designandprint/photography/imagebank.html
Disallow /computing/news/archive/
Disallow /pboro/pages/
Disallow /computing/Library/
Disallow /computing/Templates/
Disallow /admin/imago/
Disallow /service/fli/
Disallow /schools/materials/newsite/
Disallow /departments/eu/images/staff/Ruth-Kinna.jpg
Disallow /departments/eu/images/staff/Dave-Berry.jpg
Disallow /departments/eu/images/staff/Sureyya-Turkeli.jpg
Disallow /departments/eu/images/staff/Robert-Knight.jpg
Disallow /departments/materials/pg/unilever/
Disallow /departments/sses/ex/
Disallow /departments/ssehs/ex/
Disallow /departments/ssehs/information/
Disallow /service/fm/ex/
Disallow /service/publicity/vco/
Disallow /services/it/training/
Disallow /sport/sportpark/
Disallow /sport/hall-of-fame/
Disallow /news/whatsnew/
Disallow /test/
Disallow /service/studentsupportcentre/international/
Disallow /service/studentsupportcentre/international/graduate-entrepreneur.html
Disallow /students/graduation/test-webcast/
Disallow /study/portal-unavailable/
Disallow /-css/
Disallow /-images/
Disallow /study/apply/results/information-for-staff/
Disallow /study/undergraduate/2014/visiting/campustours/
Disallow /html-email/
Disallow /departments/sspgs/sspgs-dev/
Disallow /admin/ar/student/forms/
Disallow /study/clearing/register/thanks/
Disallow /departments/aed/degree-show-16/3ddesign/studentcatalogue/abigail-monger/
Disallow /services/registry/fpp/
Disallow /study/undergraduate/virtual-open-day/
Disallow /study/postgraduate/virtual-open-day/
Disallow /research/cat-irc/
Disallow /services/library-02022024/
Disallow /internal/coronavirus/latest-updates/

Warnings

  • `noindex` is not a known field.