bowdoin.edu
robots.txt
Robots Exclusion Standard data for bowdoin.edu
Resource Scan
Scan Details
| Site Domain | bowdoin.edu |
| Base Domain | bowdoin.edu |
| Scan Status | Ok |
| Last Scan | 2024-07-02T17:34:09+00:00 |
| Next Scan | 2024-08-01T17:34:09+00:00 |
Last Scan
| Scanned | 2024-07-02T17:34:09+00:00 |
| URL | https://www.bowdoin.edu/robots.txt |
| Domain IPs | 13.33.88.105, 13.33.88.41, 13.33.88.75, 13.33.88.86, 2600:9000:223b:2800:9:7591:c900:93a1, 2600:9000:223b:4800:9:7591:c900:93a1, 2600:9000:223b:9000:9:7591:c900:93a1, 2600:9000:223b:9e00:9:7591:c900:93a1, 2600:9000:223b:b200:9:7591:c900:93a1, 2600:9000:223b:c800:9:7591:c900:93a1, 2600:9000:223b:d800:9:7591:c900:93a1, 2600:9000:223b:f800:9:7591:c900:93a1 |
| Response IP | 13.33.88.41 |
| Found | Yes |
| Hash | ad36540baa4c5a5a9467a0b82f1db34a1155f1819de62cd34a68189ea22bf2a5 |
| SimHash | 14747c40c552 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /_cms/ |
| Disallow | /alerts/ |
| Disallow | /alumni-families/volunteer/volunteer-training/ |
| Disallow | /demo/ |
| Disallow | /Xhh/ |
Warnings
- `noindex` is not a known field.