Log In Sign Up

www.cs.colostate.edu
robots.txt

Robots Exclusion Standard data for www.cs.colostate.edu

Resource Scan

Scan Details

Site Domain www.cs.colostate.edu
Base Domain colostate.edu
Scan Status Ok
Last Scan2025-06-01T17:25:13+00:00
Next Scan 2025-07-01T17:25:13+00:00

Last Scan

Scanned2025-06-01T17:25:13+00:00
URL https://www.cs.colostate.edu/robots.txt
Domain IPs 129.82.45.48
Response IP 129.82.45.48
Found Yes
Hash f2776ebbcfcb9a2e78052c76f3954d834863099080eff0ac9f81194f1fcdf27d
SimHash 19861e844672

Groups

httrack

Rule Path
Disallow /
Back to top

Comments

  • robots.txt file for Colorado State University CS Department
  • Tells robot search engines (Lycos, Web Crawler, etc.) what they cannot access
Back to top

Warnings

  • 1 invalid line.
  • `` is not a known field.
Back to top