au.com
robots.txt
Robots Exclusion Standard data for au.com
Resource Scan
Scan Details
| Site Domain | au.com |
| Base Domain | au.com |
| Scan Status | Ok |
| Last Scan | 2024-04-29T18:08:58+00:00 |
| Next Scan | 2024-05-29T18:08:58+00:00 |
Last Scan
| Scanned | 2024-04-29T18:08:58+00:00 |
| URL | https://au.com/robots.txt |
| Redirect | https://www.au.com/robots.txt |
| Redirect Domain | www.au.com |
| Redirect Base | au.com |
| Domain IPs | 175.129.0.108, 2001:268:fa03:500:175:129:0:108 |
| Redirect IPs | 108.157.254.53, 108.157.254.76, 108.157.254.85, 108.157.254.89, 2600:9000:2753:1200:d:354b:4180:93a1, 2600:9000:2753:1600:d:354b:4180:93a1, 2600:9000:2753:5e00:d:354b:4180:93a1, 2600:9000:2753:600:d:354b:4180:93a1, 2600:9000:2753:9c00:d:354b:4180:93a1, 2600:9000:2753:c200:d:354b:4180:93a1, 2600:9000:2753:d600:d:354b:4180:93a1, 2600:9000:2753:e600:d:354b:4180:93a1 |
| Response IP | 108.157.254.89 |
| Found | Yes |
| Hash | c5b0f5ec682d964bb325d999611c6b24269ff52c7fc15b553a0d5ce20b4a2a8f |
| SimHash | f90cba248c93 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /cs/pay342/ |
| Disallow | /cs/pay447/ |
| Disallow | /cs/pay457/ |
| Disallow | /support/faq/included/ |
| Disallow | /*jcr%3Acontent* |
| Disallow | /*_jcr_content* |
| Disallow | /content/dam/au-com/mobile/gimon/disallow/pdf/*.pdf$ |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.au.com/sitemap.xml |
| sitemap | https://www.au.com/ipad/ipad_sitemap.xml |
| sitemap | https://www.au.com/iphone/iphone_sitemap.xml |
| sitemap | https://www.au.com/apple-watch/apple_watch_sitemap.xml |