lib.berkeley.edu
robots.txt
Robots Exclusion Standard data for lib.berkeley.edu
Resource Scan
Scan Details
| Site Domain | lib.berkeley.edu |
| Base Domain | berkeley.edu |
| Scan Status | Ok |
| Last Scan | 2024-05-17T15:29:02+00:00 |
| Next Scan | 2024-06-16T15:29:02+00:00 |
Last Scan
| Scanned | 2024-05-17T15:29:02+00:00 |
| URL | https://lib.berkeley.edu/robots.txt |
| Redirect | https://www.lib.berkeley.edu/robots.txt |
| Redirect Domain | www.lib.berkeley.edu |
| Redirect Base | berkeley.edu |
| Domain IPs | 128.32.10.243 |
| Redirect IPs | 128.32.10.243 |
| Response IP | 128.32.10.243 |
| Found | Yes |
| Hash | 0a593967ee0f3fa89e95cc450045284d7215934d1921bceb5e0067064c6b83ad |
| SimHash | e722988c0438 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /cgi-bin |
| Disallow | /data |
| Disallow | /errors |
| Disallow | videodir |
| Disallow | /Magnes |
| Disallow | /ILS/ILSrequests |
| Disallow | /images |
| Disallow | /Images |
| Disallow | /~webman |
| Disallow | /BANC/digitalscriptorium |
| Disallow | /BANC/digitalscriptorium/oldtechnical |
| Disallow | /BANC/cgi-bin |
| Disallow | /BUSI/secure |
| Disallow | /LAUC |
| Disallow | /LHRD/How |
| Disallow | /LSO |
| Disallow | /LSO/LSOweb |
| Disallow | /LSO/inventory |
| Disallow | /Reference |
| Disallow | /Staff/Admin |
| Disallow | /Staff/adminplus |
| Disallow | /Staff/archive |
| Disallow | /Staff/Asktico |
| Disallow | /Staff/billed |
| Disallow | /Staff/css |
| Disallow | /Staff/cunews-images |
| Disallow | /Staff/Doe-Moffitt |
| Disallow | /Staff/EPS |
| Disallow | /Staff/images |
| Disallow | /Staff/Innopac |
| Disallow | /Staff/instruct |
| Disallow | /Staff/javascript |
| Disallow | /Staff/js |
| Disallow | /Staff/ldo |
| Disallow | /Staff/lit/rlf_tool |
| Disallow | /Staff/mildocs |
| Disallow | /Staff/mlp |
| Disallow | /Staff/Preservation |
| Disallow | /Staff/proxy |
| Disallow | /Staff/scss |
| Disallow | /Staff/gulpfile.js |
| Disallow | /Staff/securityfilter-config_2_0.dtd |
| Disallow | /Staff/staffsearch |
| Disallow | /Staff/style.css |
| Disallow | /Staff/style.css.map |
| Disallow | /Staff/user/login |
| Disallow | /UCBonly |
| Disallow | /EART/afghanistan/ |
| Disallow | /EART/albania/ |
| Disallow | /EART/azerbijan/ |
| Disallow | /EART/britain/ |
| Disallow | /EART/colombia/ |
| Disallow | /EART/ethiopia/ |
| Disallow | /EART/french_central_africa/ |
| Disallow | /EART/georgia/ |
| Disallow | /EART/ghana/ |
| Disallow | /EART/india/ |
| Disallow | /EART/iraq/ |
| Disallow | /EART/israel/ |
| Disallow | /EART/japan/ |
| Disallow | /EART/jordan/ |
| Disallow | /EART/kenya/ |
| Disallow | /EART/kuwait/ |
| Disallow | /EART/lebanon/ |
| Disallow | /EART/qatar/ |
| Disallow | /EART/syria/ |
| Disallow | /EART/taiwan/ |
| Disallow | /EART/tajikistan/ |
| Disallow | /EART/turkey/ |
| Disallow | /EART/uae/ |
| Disallow | /EART/x-ussr/ |
| Disallow | /SSEAL/SouthAsia/cgi-bin |
| Disallow | /.svn |
| Disallow | /libstats |
| Disallow | /trac |
| Disallow | kb_upload |
| Disallow | /doemoff/dmwiki |
Comments