thewest.com.au
robots.txt
Robots Exclusion Standard data for thewest.com.au
Resource Scan
Scan Details
| Site Domain | thewest.com.au |
| Base Domain | thewest.com.au |
| Scan Status | Ok |
| Last Scan | 2024-11-09T10:06:13+00:00 |
| Next Scan | 2024-11-16T10:06:13+00:00 |
Last Scan
| Scanned | 2024-11-09T10:06:13+00:00 |
| URL | https://thewest.com.au/robots.txt |
| Domain IPs | 13.35.238.14, 13.35.238.50, 13.35.238.61, 13.35.238.77, 2600:9000:2085:3000:a:9ea7:4a40:93a1, 2600:9000:2085:5200:a:9ea7:4a40:93a1, 2600:9000:2085:7400:a:9ea7:4a40:93a1, 2600:9000:2085:8a00:a:9ea7:4a40:93a1, 2600:9000:2085:9a00:a:9ea7:4a40:93a1, 2600:9000:2085:a800:a:9ea7:4a40:93a1, 2600:9000:2085:d400:a:9ea7:4a40:93a1, 2600:9000:2085:f000:a:9ea7:4a40:93a1 |
| Response IP | 13.35.238.61 |
| Found | Yes |
| Hash | a7a53201f6cc4cb60fc20c55a11f65df531a58b1802f92621740528c087fc5e0 |
| SimHash | 6911b2146f13 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | |
| Disallow | /search |
Other Records
| Field | Value |
|---|---|
| sitemap | https://thewest.com.au/sitemap.xml |
| sitemap | https://thewest.com.au/news-sitemap.xml |