autowereld.nl
robots.txt
Robots Exclusion Standard data for autowereld.nl
Resource Scan
Scan Details
| Site Domain | autowereld.nl |
| Base Domain | autowereld.nl |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2024-04-25T21:51:21+00:00 |
| Next Scan | 2024-07-24T21:51:21+00:00 |
Last Successful Scan
| Scanned | 2023-07-01T21:47:49+00:00 |
| URL | https://autowereld.nl/robots.txt |
| Redirect | https://www.autowereld.nl/robots.txt |
| Redirect Domain | www.autowereld.nl |
| Redirect Base | autowereld.nl |
| Domain IPs | 23.59.168.120, 2600:1413:b000:6::17d5:2bd1, 2600:1413:b000:6::17d5:2bdf |
| Redirect IPs | 2600:1413:b000:6::17d5:2bd1, 2600:1413:b000:6::17d5:2bdf, 96.17.96.11, 96.17.96.23 |
| Response IP | 23.59.168.120 |
| Found | Yes |
| Hash | 5fb45a7bb4f4999b7fa653665bd07f1069fec322ec48e2d6cf3b3730828a3dec |
| SimHash | 201c09406f90 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /c?* |
| Disallow | /*meld-een-fout.html |
| Disallow | /contact.html |
| Disallow | /cdn-cgi/ |