Log In Sign Up

www.lib.berkeley.edu
robots.txt

Robots Exclusion Standard data for www.lib.berkeley.edu

Resource Scan

Scan Details

Site Domain www.lib.berkeley.edu
Base Domain berkeley.edu
Scan Status Ok
Last Scan2024-09-15T02:17:54+00:00
Next Scan 2024-10-15T02:17:54+00:00

Last Scan

Scanned2024-09-15T02:17:54+00:00
URL https://www.lib.berkeley.edu/robots.txt
Domain IPs 128.32.10.252
Response IP 128.32.10.252
Found Yes
Hash c7ebdec4734f40934a367e2a416f21c41dd09fec2dc9155aeb2deccdac9b9df8
SimHash e70a1dcd0c18

Groups

cdlwas_bot

Rule Path
Disallow

*

Rule Path
Disallow /cgi-bin
Disallow /data
Disallow /errors
Disallow /images
Disallow /Images
Disallow /BANC/bancphot
Disallow /EART/indexes
Disallow /EART/maps
Disallow /EART/tour
Disallow /EART/UCONLY
Disallow /Staff/css
Disallow /Staff/cunews-images
Disallow /Staff/images
Disallow /Staff/javascript
Disallow /Staff/js
Disallow /Staff/gulpfile.js
Disallow /Staff/securityfilter-config_2_0.dtd
Disallow /Staff/staffsearch
Disallow /Staff/style.css
Disallow /Staff/style.css.map
Disallow /Staff/user/login
Disallow /trac
Disallow kb_upload
Back to top

Comments

  • $Id: robots.txt,v 1.2 1995/10/30 04:15:40 fielding Exp $
  • robots.txt for https://www.lib.berkeley.edu/
  • see <http://www.nexor.co.uk/mak/doc/robots/norobots.html> for an explanation.
Back to top