willamette.edu
robots.txt

Robots Exclusion Standard data for willamette.edu

Resource Scan

Scan Details

Site Domain willamette.edu
Base Domain willamette.edu
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2024-09-01T22:17:58+00:00
Next Scan 2024-11-30T22:17:58+00:00

Last Successful Scan

Scanned2024-04-12T18:54:40+00:00
URL https://willamette.edu/robots.txt
Domain IPs 158.104.100.63
Response IP 158.104.100.63
Found Yes
Hash e08fc45171e4a3f4ee3d98250af6e1997e8bd481c07e4eba03b5da94db46d08f
SimHash 77d113165832

Groups

*

Rule Path
Disallow /~*/
Disallow /~jstout/
Disallow /~mgreen/gradebook.csv
Disallow /allison
Disallow /automate
Disallow /boards/trustees/php
Disallow /cgi-bin
Disallow /chatblazer
Disallow /data
Disallow /david
Disallow /emails
Disallow /emergency
Disallow /error
Disallow /local
Disallow /php
Disallow /reports
Disallow /timothy
Disallow /tools
Disallow /brian
Disallow /laura
Disallow /darick
Disallow /cla/catalog/archive/2017
Disallow /cla/catalog/archive/2016
Disallow /cla/catalog/archive/2015
Disallow /cla/catalog/archive/2014
Disallow /cla/catalog/archive/2013
Disallow /cla/catalog/archive/2012
Disallow /cla/catalog/archive/2011
Disallow /cla/catalog/archive/2010
Disallow /cla/catalog/archive/2009
Disallow /cla/catalog/archive/2008
Disallow /cla/catalog/archive/2007
Disallow /cla/catalog/archive/2006
Disallow /cla/catalog/archive/2005
Disallow /cla/catalog/archive/2004
Disallow /cla/catalog/archive/2003
Disallow /arts-sciences/catalog/archive/2017
Disallow /arts-sciences/catalog/archive/2016
Disallow /arts-sciences/catalog/archive/2015
Disallow /arts-sciences/catalog/archive/2014
Disallow /arts-sciences/catalog/archive/2013
Disallow /arts-sciences/catalog/archive/2012
Disallow /arts-sciences/catalog/archive/2011
Disallow /arts-sciences/catalog/archive/2010
Disallow /arts-sciences/catalog/archive/2009
Disallow /arts-sciences/catalog/archive/2008
Disallow /arts-sciences/catalog/archive/2007
Disallow /arts-sciences/catalog/archive/2006
Disallow /arts-sciences/catalog/archive/2005
Disallow /arts-sciences/catalog/archive/2004
Disallow /arts-sciences/catalog/archive/2003
Disallow /support/recognition/honor_roll
Disallow /dept/fa/svpfa
Disallow /_BREI
Disallow /asp
Disallow /alumni/community/classnotes
Disallow /community/icl/more-about-icl/members/

Other Records

Field Value
sitemap https://willamette.edu/sitemap.xml