discount.com.au
robots.txt
Robots Exclusion Standard data for discount.com.au
Resource Scan
Scan Details
| Site Domain | discount.com.au |
| Base Domain | discount.com.au |
| Scan Status | Ok |
| Last Scan | 2024-11-16T08:07:34+00:00 |
| Next Scan | 2024-12-16T08:07:34+00:00 |
Last Scan
| Scanned | 2024-11-16T08:07:34+00:00 |
| URL | https://discount.com.au/robots.txt |
| Domain IPs | 104.26.10.187, 104.26.11.187, 172.67.75.128, 2606:4700:20::681a:abb, 2606:4700:20::681a:bbb, 2606:4700:20::ac43:4b80 |
| Response IP | 104.26.10.187 |
| Found | Yes |
| Hash | b72384d6aa81e10c91626c67b95b56d885c90de343958b6e5b8645c24c44ff4d |
| SimHash | e9584970e631 |
Groups
*
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin/ | - |
| Disallow | /wp-admin/ | - |
| Disallow | /wp-includes/ | - |
| Disallow | /trackback/ | - |
| Disallow | /feed/ | - |
| Disallow | /comments/ | - |
| Disallow | /*?blackhole | - |
| Disallow | /?blackhole | - |
| Disallow | /*? | Block parameterized URLs, but ensure critical URLs using parameters aren't blocked |
| Allow | /wp-content/uploads/ | - |
| Allow | /wp-includes/css/ | - |
| Allow | /wp-includes/js/ | - |
| Allow | /wp-includes/images/ | - |
| Allow | /wp-content/plugins/ | - |
| Allow | /wp-content/themes/ | - |
Other Records
| Field | Value |
|---|---|
| sitemap | https://discount.com.au/sitemap_index.xml |
Comments