ssec.wisc.edu
robots.txt

Robots Exclusion Standard data for ssec.wisc.edu

Resource Scan

Scan Details

Site Domain ssec.wisc.edu
Base Domain wisc.edu
Scan Status Ok
Last Scan2025-05-25T06:23:20+00:00
Next Scan 2025-06-24T06:23:20+00:00

Last Scan

Scanned2025-05-25T06:23:20+00:00
URL https://ssec.wisc.edu/robots.txt
Redirect https://www.ssec.wisc.edu/robots.txt
Redirect Domain www.ssec.wisc.edu
Redirect Base wisc.edu
Domain IPs 128.104.111.211
Redirect IPs 128.104.111.211
Response IP 128.104.111.211
Found Yes
Hash 3f54b382130ba314ff74a191ac375904f98a5707717b297a988553a0384e3f6b
SimHash ac1039599d31

Groups

*

Rule Path
Disallow /cgi-bin/email_form.cgi
Disallow /employee_info
Disallow /accounting
Disallow /site_admin
Disallow /register
Disallow /pay

Comments

  • /robots.txt file for http://www.ssec.wisc.edu/
  • disallow the following, including employee_info, which
  • is password protected already and shouldn't be crawled,
  • but is added for redundancy