www.rotman.utoronto.ca
robots.txt
Robots Exclusion Standard data for www.rotman.utoronto.ca
Resource Scan
Scan Details
| Site Domain | www.rotman.utoronto.ca |
| Base Domain | utoronto.ca |
| Scan Status | Ok |
| Last Scan | 2025-07-02T01:40:18+00:00 |
| Next Scan | 2025-08-01T01:40:18+00:00 |
Last Scan
| Scanned | 2025-07-02T01:40:18+00:00 |
| URL | https://www.rotman.utoronto.ca/robots.txt |
| Domain IPs | 15.157.79.226, 3.97.145.103 |
| Response IP | 15.157.79.226 |
| Found | Yes |
| Hash | c086d3ba609d58cdb379c48f0c1dc9f378a8a137759a43a33cffbde59f07c9a0 |
| SimHash | 6010dd536731 |
Groups
terminalfour nutch spider
No rules defined. All paths allowed.
Other Records
| Field | Value |
|---|---|
| crawl-delay | 0.5 |
searchstax crawler/1.0 (+https://www.searchstax.com)
| Rule | Path |
|---|---|
| Disallow | /search/ |
| Disallow | /site-assets/ |
| Disallow | /component-library/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 0.5 |
*
| Rule | Path |
|---|---|
| Disallow | / |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.rotman.utoronto.ca/sitemap-en.xml |