autotrader.co.za
robots.txt
Robots Exclusion Standard data for autotrader.co.za
Resource Scan
Scan Details
Site Domain | autotrader.co.za |
Base Domain | autotrader.co.za |
Scan Status | Ok |
Last Scan | 2024-09-15T11:42:55+00:00 |
Next Scan | 2024-09-29T11:42:55+00:00 |
Last Scan
Scanned | 2024-09-15T11:42:55+00:00 |
URL | https://autotrader.co.za/robots.txt |
Redirect | https://www.autotrader.co.za/robots.txt |
Redirect Domain | www.autotrader.co.za |
Redirect Base | autotrader.co.za |
Domain IPs | 80.88.11.43 |
Redirect IPs | 80.88.11.43 |
Response IP | 80.88.11.43 |
Found | Yes |
Hash | 35c76569100ad5526641d55f764e9cd98d69892adbb4e0d08633adf835182bfb |
SimHash | 034e585ac913 |
Groups
*
Rule | Path |
---|---|
Disallow | /answers/Account |
Disallow | /answers/bundles |
Disallow | /answers/Comment |
Disallow | /answers/Content |
Disallow | /answers/DownloadFile.aspx |
Disallow | /answers/file |
Disallow | /answers/images |
Disallow | /answers/javascript |
Disallow | /answers/Question |
Disallow | /RecentlyViewed |
Other Records
Field | Value |
---|---|
sitemap | https://www.autotrader.co.za/sitemap/index |