Log In Sign Up

vw.com.mx
robots.txt

Robots Exclusion Standard data for vw.com.mx

Resource Scan

Scan Details

Site Domain vw.com.mx
Base Domain vw.com.mx
Scan Status Ok
Last Scan2025-02-16T18:02:11+00:00
Next Scan 2025-03-18T18:02:11+00:00

Last Scan

Scanned2025-02-16T18:02:11+00:00
URL https://vw.com.mx/robots.txt
Redirect https://www.vw.com.mx:443/robots.txt
Redirect Domain www.vw.com.mx
Redirect Base vw.com.mx
Domain IPs 75.2.18.28
Redirect IPs 23.45.207.164, 23.45.207.165, 2600:1413:b000:13::b857:c193, 2600:1413:b000:13::b857:c195
Response IP 42.99.140.219
Found Yes
Hash a0b0b7b20c7a8acc6cc7f45a0ce684d4d017e86578f537df64ee659152435b53
SimHash 69289ac5e7d3

Groups

googlebot

Rule Path
Allow /app/
Disallow /*notification
Disallow /content/mx_mexico/
Disallow /es/tools/navigation/meta/busqueda.html
Disallow /es/tools/newlightbox/
Disallow /_jcr_content/lightbox/
Disallow /_jcr_content/flexparsys/
Disallow /content/dam/vw-ngw/vw_pkw/importers/mx/descargables/
Disallow /app/concesionarias/vw-mx/es
Disallow /*.integrator-includes.txt
Disallow /*.integrator-includes.json

slurp

Rule Path
Allow /app/
Disallow /*.integrator-includes.txt
Disallow /*.integrator-includes.json

bingbot

Rule Path
Allow /app/
Disallow /*.integrator-includes.txt
Disallow /*.integrator-includes.json

msnbot

Rule Path
Allow /app/
Disallow /*.integrator-includes.txt
Disallow /*.integrator-includes.json

*

Rule Path
Disallow /app/
Disallow /*notification
Disallow /content
Disallow /etc
Disallow /*.integrator-includes.txt
Disallow /*.integrator-includes.json
Back to top

Other Records

Field Value
sitemap https://www.vw.com.mx/es.sitemap.xml
sitemap https://www.vw.com.mx/idhub/content/dam/technical/sitemap/onehub-MX-experiencia-showroom-apps-sitemap.xml
Back to top

Comments

  • whitelist user agents / pages to visit for app-paths
  • default block
  • sitemap
Back to top