volkswagen.be
robots.txt

Robots Exclusion Standard data for volkswagen.be

Resource Scan

Scan Details

Site Domain volkswagen.be
Base Domain volkswagen.be
Scan Status Ok
Last Scan2025-03-23T18:13:55+00:00
Next Scan 2025-04-22T18:13:55+00:00

Last Scan

Scanned2025-03-23T18:13:55+00:00
URL https://volkswagen.be/robots.txt
Redirect https://www.volkswagen.be:443/robots.txt
Redirect Domain www.volkswagen.be
Redirect Base volkswagen.be
Domain IPs 75.2.74.46
Redirect IPs 23.209.46.82, 23.209.46.89, 2600:1413:b000:13::b857:c18c, 2600:1413:b000:13::b857:c19b
Response IP 42.99.140.146
Found Yes
Hash 2439fcc500a8b0c295462cd7c1448c57878766f700a6d9d7c0504f8e28f2add6
SimHash 4c2892cdcf95

Groups

googlebot

Rule Path
Allow /app/
Disallow /*.integrator-includes.txt
Disallow /*.integrator-includes.json

slurp

Rule Path
Allow /app/
Disallow /*.integrator-includes.txt
Disallow /*.integrator-includes.json

bingbot

Rule Path
Allow /app/
Disallow /*.integrator-includes.txt
Disallow /*.integrator-includes.json

msnbot

Rule Path
Allow /app/
Disallow /*.integrator-includes.txt
Disallow /*.integrator-includes.json

*

Rule Path
Disallow /*notification
Disallow /*.integrator-includes.txt
Disallow /*.integrator-includes.json
Disallow /app/shop/fr/panier
Disallow /app/shop/fr/Confirme
Disallow /app/shop/fr/Step2-confirm
Disallow /app/shop/fr/Step3-confirme
Disallow /app/shop/fr/Order
Disallow /app/shop/nl/winkelwagen
Disallow /app/shop/nl/confirme
Disallow /app/shop/nl/Step2-confirm
Disallow /app/shop/nl/Step3-confirme
Disallow /app/shop/nl/Order

Other Records

Field Value
sitemap https://www.volkswagen.be/idhub/content/dam/technical/sitemap/onehub-be-nl-temp-sitemap-PC.xml
sitemap https://www.volkswagen.be/idhub/content/dam/technical/sitemap/onehub-be-fr-temp-sitemap-PC.xml
sitemap https://www.volkswagen.be/idhub/content/dam/technical/sitemap/onehub-be-nl-showrooms-PC.xml
sitemap https://www.volkswagen.be/idhub/content/dam/technical/sitemap/onehub-be-fr-showrooms-PC.xml
sitemap https://www.volkswagen.be/app/shop/fr/sitemap/statics
sitemap https://www.volkswagen.be/app/shop/nl/sitemap/statics
sitemap https://www.volkswagen.be/app/shop/fr/sitemap/product
sitemap https://www.volkswagen.be/app/shop/nl/sitemap/product
sitemap https://www.volkswagen.be/app/shop/fr/sitemap/categories
sitemap https://www.volkswagen.be/app/shop/nl/sitemap/categories

Comments

  • whitelist user agents / pages to visit for app-paths
  • default block
  • sitemap