ben.edu
robots.txt
Robots Exclusion Standard data for ben.edu
Resource Scan
Scan Details
| Site Domain | ben.edu |
| Base Domain | ben.edu |
| Scan Status | Ok |
| Last Scan | 2024-09-19T10:48:46+00:00 |
| Next Scan | 2024-10-19T10:48:46+00:00 |
Last Scan
| Scanned | 2024-09-19T10:48:46+00:00 |
| URL | https://ben.edu/robots.txt |
| Domain IPs | 141.193.213.20, 141.193.213.21 |
| Response IP | 141.193.213.20 |
| Found | Yes |
| Hash | 31ee4a1ba118dd253d120707a3a2b4a4de6cffc70fc0569b199be25f27cc6373 |
| SimHash | 893888c2a605 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /search/ |
| Disallow | /calendar/action* |
| Disallow | /events/action* |
| Allow | /*.css |
| Allow | /*.js |
| Disallow | /*? |
| Disallow | /benu-event* |
Other Records
| Field | Value |
|---|---|
| sitemap | https://ben.edu/sitemap.xml |
Warnings
- 4 invalid lines.