rotman.utoronto.ca
robots.txt
Robots Exclusion Standard data for rotman.utoronto.ca
Resource Scan
Scan Details
| Site Domain | rotman.utoronto.ca |
| Base Domain | utoronto.ca |
| Scan Status | Ok |
| Last Scan | 2025-06-19T20:49:34+00:00 |
| Next Scan | 2025-07-19T20:49:34+00:00 |
Last Scan
| Scanned | 2025-06-19T20:49:34+00:00 |
| URL | https://rotman.utoronto.ca/robots.txt |
| Domain IPs | 15.197.164.135, 99.83.211.213 |
| Response IP | 99.83.211.213 |
| Found | Yes |
| Hash | c086d3ba609d58cdb379c48f0c1dc9f378a8a137759a43a33cffbde59f07c9a0 |
| SimHash | 6010dd536731 |
Groups
terminalfour nutch spider
No rules defined. All paths allowed.
Other Records
| Field | Value |
|---|---|
| crawl-delay | 0.5 |
searchstax crawler/1.0 (+https://www.searchstax.com)
| Rule | Path |
|---|---|
| Disallow | /search/ |
| Disallow | /site-assets/ |
| Disallow | /component-library/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 0.5 |
*
| Rule | Path |
|---|---|
| Disallow | / |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.rotman.utoronto.ca/sitemap-en.xml |