autotrader.nl
robots.txt
Robots Exclusion Standard data for autotrader.nl
Resource Scan
Scan Details
Site Domain | autotrader.nl |
Base Domain | autotrader.nl |
Scan Status | Ok |
Last Scan | 2024-06-22T02:57:58+00:00 |
Next Scan | 2024-06-29T02:57:58+00:00 |
Last Scan
Scanned | 2024-06-22T02:57:58+00:00 |
URL | https://autotrader.nl/robots.txt |
Redirect | https://www.autotrader.nl/robots.txt |
Redirect Domain | www.autotrader.nl |
Redirect Base | autotrader.nl |
Domain IPs | 3.165.102.116, 3.165.102.20, 3.165.102.55, 3.165.102.85 |
Redirect IPs | 108.156.133.121, 108.156.133.17, 108.156.133.19, 108.156.133.82 |
Response IP | 108.156.133.82 |
Found | Yes |
Hash | 8204e654b25c7eb2440168103080b4e86d41a5770e924f3c04d55597ff2f81f0 |
SimHash | 891aeaaed1bb |
Groups
*
Rule | Path |
---|---|
Disallow | /auto/mijn-trader/addtocompareradverts |
Disallow | /auto/mijn-trader/removefromcompareradverts |
Disallow | /auto/mijn-trader/compareradvertspopup |
Disallow | /motor/mijn-trader/addtocompareradverts |
Disallow | /motor/mijn-trader/removefromcompareradverts |
Disallow | /motor/mijn-trader/compareradvertspopup |
Disallow | /bedrijfswagen/mijn-trader/addtocompareradverts |
Disallow | /bedrijfswagen/mijn-trader/removefromcompareradverts |
Disallow | /bedrijfswagen/mijn-trader/compareradvertspopup |
Disallow | /klassieker/mijn-trader/addtocompareradverts |
Disallow | /klassieker/mijn-trader/removefromcompareradverts |
Disallow | /klassieker/mijn-trader/compareradvertspopup |
Disallow | /api/ |
Disallow | /transformCookie |
Allow | / |
Comments