study.sydney
robots.txt
Robots Exclusion Standard data for study.sydney
Resource Scan
Scan Details
| Site Domain | study.sydney |
| Base Domain | study.sydney |
| Scan Status | Ok |
| Last Scan | 2024-06-20T06:25:24+00:00 |
| Next Scan | 2024-07-20T06:25:24+00:00 |
Last Scan
| Scanned | 2024-06-20T06:25:24+00:00 |
| URL | https://study.sydney/robots.txt |
| Redirect | https://www.study.nsw.gov.au/robots.txt |
| Redirect Domain | www.study.nsw.gov.au |
| Redirect Base | nsw.gov.au |
| Domain IPs | 43.245.43.3 |
| Redirect IPs | 104.18.12.46, 104.18.13.46, 2606:4700::6812:c2e, 2606:4700::6812:d2e |
| Response IP | 104.18.12.46 |
| Found | Yes |
| Hash | 5693411b67f51b2cf0b3c393fd6d5a0150ec043b7cb87bb8cccd8b4c0949962f |
| SimHash | 484ddc10c4b4 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /admin/ |
| Disallow | /dev/ |
| Disallow | /Security/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.study.nsw.gov.au/sitemap.xml |
Warnings
- 1 invalid line.
- `request-rate` is not a known field.