schoenentorfs.be
robots.txt
Robots Exclusion Standard data for schoenentorfs.be
Resource Scan
Scan Details
Site Domain | schoenentorfs.be |
Base Domain | schoenentorfs.be |
Scan Status | Ok |
Last Scan | 2025-02-22T11:54:40+00:00 |
Next Scan | 2025-03-24T11:54:40+00:00 |
Last Scan
Scanned | 2025-02-22T11:54:40+00:00 |
URL | https://www.schoenentorfs.be/robots.txt |
Redirect | https://www.torfs.be/robots.txt |
Redirect Domain | www.torfs.be |
Redirect Base | torfs.be |
Domain IPs | 104.18.22.15, 104.18.23.15, 2606:4700::6812:160f, 2606:4700::6812:170f |
Redirect IPs | 104.18.4.129, 104.18.5.129, 2606:4700::6812:481, 2606:4700::6812:581 |
Response IP | 104.18.5.129 |
Found | Yes |
Hash | 2f7318314198f6d22876ec8bfd6dd6b29cf94a1b993d96e6ed43b9e6edb7bce9 |
SimHash | 4279d1606362 |
Groups
*
Rule | Path |
---|---|
Disallow | /on/demandware.store/* |
Disallow | *Login-* |
Disallow | *Register-* |
Disallow | *Wishlist-* |
Disallow | *Cart-* |
Disallow | *Checkout-* |
Disallow | *Account-* |
Disallow | *Order-* |
Disallow | /*Login-OAuthLogin |
Disallow | /*MolliePayment-Redirect |
Disallow | /mijn-account |
Disallow | /winkelmandje |
Disallow | /checkout |
Disallow | /verlanglijstje |
Disallow | /overzicht |
Disallow | /Activeer |
Disallow | /mon-compte |
Disallow | /mon-panier |
Disallow | /mes-coups-de-coeur |
Disallow | /Activer |
Disallow | /apercu |
Disallow | /*null* |
Disallow | /*/passwordreset/ |
Disallow | /*/account/ |
Disallow | /*/orders/ |
Disallow | /*/profile/ |
Allow | /*?dwvar |
Allow | /*utm_medium%3Dmerchant |
Disallow | /*pmin* |
Disallow | /*pmax* |
Disallow | /*srule%3D* |
Disallow | /*?q=* |
Disallow | /*prefv*%3D* |
Disallow | /*prefn*%3D* |
Disallow | /*%26sz%3D |
Allow | /*channable%3D* |
msnbot
slurp
Rule | Path |
---|---|
Disallow | /on/demandware.store/* |
Disallow | *Login-* |
Disallow | *Register-* |
Disallow | *Wishlist-* |
Disallow | *Cart-* |
Disallow | *Checkout-* |
Disallow | *Account-* |
Disallow | *Order-* |
Disallow | /*Login-OAuthLogin |
Disallow | /*MolliePayment-Redirect |
Disallow | /mijn-account |
Disallow | /winkelmandje |
Disallow | /checkout |
Disallow | /verlanglijstje |
Disallow | /overzicht |
Disallow | /Activeer |
Disallow | /mon-compte |
Disallow | /mon-panier |
Disallow | /mes-coups-de-coeur |
Disallow | /Activer |
Disallow | /apercu |
Disallow | /*null* |
Disallow | /*/passwordreset/ |
Disallow | /*/account/ |
Disallow | /*/orders/ |
Disallow | /*/profile/ |
Allow | /*?dwvar |
Allow | /*utm_medium%3Dmerchant |
Disallow | /*pmin* |
Disallow | /*pmax* |
Disallow | /*srule%3D* |
Disallow | /*?q=* |
Disallow | /*prefv*%3D* |
Disallow | /*prefn*%3D* |
Disallow | /*%26sz%3D |
Allow | /*channable%3D* |
Other Records
Field | Value |
---|---|
crawl-delay | 30 |
ahrefsbot
rogerbot
Rule | Path |
---|---|
Disallow | /on/demandware.store/* |
Disallow | *Login-* |
Disallow | *Register-* |
Disallow | *Wishlist-* |
Disallow | *Cart-* |
Disallow | *Checkout-* |
Disallow | *Account-* |
Disallow | *Order-* |
Disallow | /*Login-OAuthLogin |
Disallow | /*MolliePayment-Redirect |
Disallow | /mijn-account |
Disallow | /winkelmandje |
Disallow | /checkout |
Disallow | /verlanglijstje |
Disallow | /overzicht |
Disallow | /Activeer |
Disallow | /mon-compte |
Disallow | /mon-panier |
Disallow | /mes-coups-de-coeur |
Disallow | /Activer |
Disallow | /apercu |
Disallow | /*null* |
Disallow | /*/passwordreset/ |
Disallow | /*/account/ |
Disallow | /*/orders/ |
Disallow | /*/profile/ |
Allow | /*?dwvar |
Allow | /*utm_medium%3Dmerchant |
Disallow | /*pmin* |
Disallow | /*pmax* |
Disallow | /*srule%3D* |
Disallow | /*?q=* |
Disallow | /*prefv*%3D* |
Disallow | /*prefn*%3D* |
Disallow | /*%26sz%3D |
Allow | /*channable%3D* |
Other Records
Field | Value |
---|---|
crawl-delay | 120 |
Other Records
Field | Value |
---|---|
sitemap | https://www.torfs.be/sitemap_index.xml |
sitemap | https://www.torfs.be/nl/vp/nc/sitemap/keyword.xml |
sitemap | https://www.torfs.be/fr/vp/nc/sitemap/keyword.xml |
Warnings
- 2 invalid lines.
Comments