goto.msstate.edu
robots.txt

Robots Exclusion Standard data for goto.msstate.edu

Resource Scan

Scan Details

Site Domain goto.msstate.edu
Base Domain msstate.edu
Scan Status Ok
Last Scan2024-06-18T15:29:27+00:00
Next Scan 2024-07-02T15:29:27+00:00

Last Scan

Scanned2024-06-18T15:29:27+00:00
URL https://goto.msstate.edu/robots.txt
Domain IPs 34.198.122.35
Response IP 34.198.122.35
Found Yes
Hash ff8bf112376816561233a51f27de3f30490ed2d505cfdf71e68a65c26e2ea26b
SimHash 298835f24732

Groups

*

Rule Path
Disallow /

Comments

  • /robots.txt file for https://goto.msstate.edu
  • dont allow crawling of anything
  • Disallow: /apply/
  • Disallow: /fonts/
  • Disallow: /images/
  • Disallow: /shared/