dustin.nl
robots.txt
Robots Exclusion Standard data for dustin.nl
Resource Scan
Scan Details
| Site Domain | dustin.nl |
| Base Domain | dustin.nl |
| Scan Status | Ok |
| Last Scan | 2026-03-20T19:03:02+00:00 |
| Next Scan | 2026-04-03T19:03:02+00:00 |
Last Scan
| Scanned | 2026-03-20T19:03:02+00:00 |
| URL | https://dustin.nl/robots.txt |
| Redirect | https://www.dustin.nl/robots.txt |
| Redirect Domain | www.dustin.nl |
| Redirect Base | dustin.nl |
| Domain IPs | 104.18.24.89, 104.18.25.89, 2606:4700::6812:1859, 2606:4700::6812:1959 |
| Redirect IPs | 104.18.24.89, 104.18.25.89, 2606:4700::6812:1859, 2606:4700::6812:1959 |
| Response IP | 104.18.24.89 |
| Found | Yes |
| Hash | 5118cad91798b5747ea8eb98ea476c694e395634ce4e95d2640d626aa8a04b89 |
| SimHash | eea9f97385e2 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /cases |
| Disallow | /compare |
| Disallow | /registration |
| Disallow | /order |
| Disallow | /login |
| Disallow | /account |
| Disallow | /checkout |
| Disallow | /shoppingcart |
| Disallow | /favorites |
| Disallow | /accessories |
| Disallow | /*?tduid=* |
| Disallow | /*?tt=* |
| Disallow | /setlanguage |
| Disallow | /attestcheckout |
| Disallow | /preorder |
| Disallow | /dustinapi |
| Disallow | /rma |
| Disallow | /api/rma |
| Disallow | /api/logger/newrelic |
| Disallow | /maintenance |
| Disallow | /testvarumaerken |
| Disallow | /testvaremaerker |
| Disallow | /testvaremerker |
| Disallow | /testivalmistajat |
| Disallow | /test |
| Disallow | /testi |
| Disallow | /solutions/test-kunskapsbanken |
| Disallow | /solutions/test-vidensbanken |
| Disallow | /tjenester/test-kunnskapsbanken |
| Disallow | /solutions/testi-tietopankki |
| Disallow | /test-kennisbank |
| Disallow | /ShoppingCart/GetMiniCartPreviewAjax |
| Disallow | /dacsaportal |
| Disallow | /DacsaPortal |
| Disallow | /pd_* |
| Disallow | /*categorypath%3D* |
| Disallow | /*tags%3D* |
| Disallow | /*tips%3D* |
| Disallow | /graphql |
| Disallow | /contentgraphql |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.dustin.nl/product-sitemap.xml |
| sitemap | https://www.dustin.nl/category-sitemap.xml |
| sitemap | https://www.dustin.nl/content-sitemap.xml |
| sitemap | https://www.dustin.nl/product-eol-sitemap.xml |