vw.com
robots.txt

Robots Exclusion Standard data for vw.com

Resource Scan

Scan Details

Site Domain vw.com
Base Domain vw.com
Scan Status Ok
Last Scan2025-03-20T04:26:16+00:00
Next Scan 2025-04-19T04:26:16+00:00

Last Scan

Scanned2025-03-20T04:26:16+00:00
URL https://vw.com/robots.txt
Redirect https://www.vw.com:443/robots.txt
Redirect Domain www.vw.com
Redirect Base vw.com
Domain IPs 75.2.18.28, 99.83.199.128
Redirect IPs 184.87.193.142, 184.87.193.154, 2600:1413:b000:13::b857:c18e, 2600:1413:b000:13::b857:c19a
Response IP 23.45.207.167
Found Yes
Hash 4f27c873b4ed0232844aeb7d2af8fee1947a0fc7ceea8c78acc28d90d8ccb583
SimHash 69689ac4cdd5

Groups

googlebot

Rule Path
Allow /app/

slurp

Rule Path
Allow /app/

bingbot

Rule Path
Allow /app/

msnbot

Rule Path
Allow /app/

*

Rule Path
Disallow /app/
Disallow /*notification
Disallow /*.integrator-includes.txt
Disallow /*.integrator-includes.json

Other Records

Field Value
sitemap https://www.vw.com/en.sitemap.xml
sitemap https://www.vw.com/idhub/content/dam/technical/sitemap/us.es.sitemap.xml

Comments

  • ROBOTS TXT
  • whitelist user agents / pages to visit for app-paths
  • default block
  • sitemap
  • sitemap es