open.umn.edu
robots.txt

Robots Exclusion Standard data for open.umn.edu

Resource Scan

Scan Details

Site Domain open.umn.edu
Base Domain umn.edu
Scan Status Ok
Last Scan2025-08-30T13:25:43+00:00
Next Scan 2025-09-29T13:25:43+00:00

Last Scan

Scanned2025-08-30T13:25:43+00:00
URL https://open.umn.edu/robots.txt
Domain IPs 134.84.72.28
Response IP 134.84.72.28
Found Yes
Hash 74f1049de2e97997007697bc5900092352e4331d6560a394cc4b4361c9f64422
SimHash 4a0ccc5083f1

Groups

*

Rule Path
Allow /

Other Records

Field Value
crawl-delay 120

bytespider | byte spider | bytedance

Rule Path
Disallow /

Other Records

Field Value
sitemap https://open.umn.edu/sitemap.xml.gz