sits.london.ac.uk
robots.txt

Robots Exclusion Standard data for sits.london.ac.uk

Resource Scan

Scan Details

Site Domain sits.london.ac.uk
Base Domain london.ac.uk
Scan Status Ok
Last Scan2024-11-03T08:15:55+00:00
Next Scan 2024-12-03T08:15:55+00:00

Last Scan

Scanned2024-11-03T08:15:55+00:00
URL https://sits.london.ac.uk/robots.txt
Domain IPs 13.107.246.59, 2620:1ec:bdf::59
Response IP 13.107.246.59
Found Yes
Hash 6ce84db2e0ed008d8d025ce974108ecfa6fe1758c84e49e910317f64567eca1c
SimHash 4d50dd32c3b0

Groups

*
adsbot-google
adsbot-google-mobile
adsbot-google-mobile-apps

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

Comments

  • Disallow: /urd/
  • Disallow: /srd/