spaargids.be
robots.txt
Robots Exclusion Standard data for spaargids.be
Resource Scan
Scan Details
| Site Domain | spaargids.be |
| Base Domain | spaargids.be |
| Scan Status | Ok |
| Last Scan | 2024-11-07T21:38:33+00:00 |
| Next Scan | 2024-11-14T21:38:33+00:00 |
Last Scan
| Scanned | 2024-11-07T21:38:33+00:00 |
| URL | https://spaargids.be/robots.txt |
| Redirect | https://www.spaargids.be/robots.txt |
| Redirect Domain | www.spaargids.be |
| Redirect Base | spaargids.be |
| Domain IPs | 23.32.29.88, 23.32.29.90, 2600:1413:b000:6::17d5:2bc5, 2600:1413:b000:6::17d5:2bdf |
| Redirect IPs | 23.32.29.88, 23.32.29.90, 2600:1413:b000:6::17d5:2bc5, 2600:1413:b000:6::17d5:2bdf |
| Response IP | 23.59.168.138 |
| Found | Yes |
| Hash | 1d8b5d22af19d2d8f4bcfb5906cbf4fb39aecee02f08ff41dd2a381bb807290c |
| SimHash | 644c62524db5 |
Groups
*
| Rule | Path |
|---|---|
| Allow | /*.js* |
| Allow | /*.css* |
| Allow | /*.png* |
| Allow | /*.jpg* |
| Allow | /*.gif* |
| Disallow | /sparen/log-zelf-in.html |
| Disallow | /sparen/maak-een-nieuw-account.html |
| Disallow | /sparen/mijn-autoverzekering-offertes.html |
| Disallow | /forum/search.php |
| Disallow | *com_weblinks* |
| Disallow | /sparen/infofiche.html* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 30 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.spaargids.be/sparen/sitemap/xml |
| sitemap | https://www.spaargids.be/forum/sitemap.xml |