stonybrook.edu
robots.txt
Robots Exclusion Standard data for stonybrook.edu
Resource Scan
Scan Details
| Site Domain | stonybrook.edu |
| Base Domain | stonybrook.edu |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2024-09-22T14:00:28+00:00 |
| Next Scan | 2024-11-21T14:00:28+00:00 |
Last Successful Scan
| Scanned | 2024-03-04T13:57:59+00:00 |
| URL | https://stonybrook.edu/robots.txt |
| Redirect | https://www.stonybrook.edu/robots.txt |
| Redirect Domain | www.stonybrook.edu |
| Redirect Base | stonybrook.edu |
| Domain IPs | 104.18.6.126, 104.18.7.126, 2606:4700::6812:67e, 2606:4700::6812:77e |
| Redirect IPs | 104.18.6.126, 104.18.7.126, 2606:4700::6812:67e, 2606:4700::6812:77e |
| Response IP | 104.18.6.126 |
| Found | Yes |
| Hash | e9c521e5506a5b9ba79be34d916ddb9ba6b2f6f181dfa367e49bb81c6494e532 |
| SimHash | 985c87b8a83f |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /globality/HansenGSJ78StagingArea/ |
| Disallow | /ugrdbulletin/2009-11/fall2009/index.shtml |
| Disallow | /ugrdbulletin/2007-09/spring2009/index.shtml |
| Disallow | /ugrdbulletin/2007-09/fall2008/index.shtml |
| Disallow | /ugrdbulletin/2007-09/spring2008/index.shtml |
| Disallow | /ugrdbulletin/2007-09/fall2007/index.shtml |
| Disallow | /ugrdbulletin/2007-09/print/index.shtml |
| Disallow | /ugrdbulletin/2005-07/spring2007/index.shtml |
| Disallow | /ugrdbulletin/2005-07/fall2006/index.shtml |
| Disallow | /ugrdbulletin/2005-07/spring2006/index.shtml |
| Disallow | /ugrdbulletin/2005-07/fall2005/index.shtml |
| Disallow | /ugrdbulletin/2005-07/print/index.shtml |
| Disallow | /hsc-bulletin/ |
| Disallow | /test/ |
| Disallow | /tools/ |
| Disallow | /top-stories-2015/index-dev.php |
| Disallow | /mobile/ |
| Disallow | /request-assistance/ |
| Disallow | /faculty-directory/ |
| Disallow | /analytics/ |
| Disallow | /thank-you/ |
| Disallow | /tutorials/ |
| Disallow | /shared/ |
| Disallow | /resources/ |
| Disallow | /plugins/ |
| Disallow | /elements/ |
| Disallow | /tools/ |
| Disallow | /_archive/ |
| Disallow | /commcms/_archive/ |
| Disallow | /_test/ |
| Disallow | /commcms/_test/ |
| Disallow | /_old/ |
| Disallow | /commcms/_old/ |
| Disallow | /commcms/grad/_resources/ |