network.justauto.com.au
robots.txt
Robots Exclusion Standard data for network.justauto.com.au
Resource Scan
Scan Details
| Site Domain | network.justauto.com.au |
| Base Domain | justauto.com.au |
| Scan Status | Ok |
| Last Scan | 2024-10-01T06:23:08+00:00 |
| Next Scan | 2024-10-31T06:23:08+00:00 |
Last Scan
| Scanned | 2024-10-01T06:23:08+00:00 |
| URL | https://network.justauto.com.au/robots.txt |
| Domain IPs | 104.18.10.35, 104.18.11.35, 2606:4700::6812:a23, 2606:4700::6812:b23 |
| Response IP | 104.18.11.35 |
| Found | Yes |
| Hash | 1dd5b8f82f46025148544ff7ec32cabd9987e49ecb99c90a592904e11d9e1414 |
| SimHash | 49a294cc0690 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /dashboard/ |
| Disallow | /info/ |
| Disallow | /user/login/ |
| Disallow | /flag/flag/favourite_ad/* |
| Disallow | /flag/unflag/favourite_ad/* |
| Disallow | */show-number |
| Disallow | /node/ |
| Disallow | /node/add/ |
| Disallow | /subscribe/confirmation |
| Disallow | /webhook/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 2 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://network.justauto.com.au/sitemap.xml |
Comments