Log In Sign Up

volkswagen.no
robots.txt

Robots Exclusion Standard data for volkswagen.no

Resource Scan

Scan Details

Site Domain volkswagen.no
Base Domain volkswagen.no
Scan Status Ok
Last Scan2025-03-30T11:03:08+00:00
Next Scan 2025-04-29T11:03:08+00:00

Last Scan

Scanned2025-03-30T11:03:08+00:00
URL https://volkswagen.no/robots.txt
Redirect https://www.volkswagen.no:443/robots.txt
Redirect Domain www.volkswagen.no
Redirect Base volkswagen.no
Domain IPs 75.2.48.58
Redirect IPs 23.46.230.137, 23.46.230.146, 2600:1413:b000:13::b857:c18b, 2600:1413:b000:13::b857:c19e
Response IP 23.52.171.139
Found Yes
Hash 2e29b62864ca4dbe25e777c8c376b31da9d97c54f2a1c81ccab7b2fd371b031c
SimHash 242293d5c5f7

Groups

googlebot

Rule Path
Allow /app/
Disallow /*.integrator-includes.txt
Disallow /*.integrator-includes.json
Disallow /*.forhandlere-bilverksted.*/

slurp

Rule Path
Allow /app/
Disallow /*.integrator-includes.txt
Disallow /*.integrator-includes.json
Disallow /*.forhandlere-bilverksted.*/

bingbot

Rule Path
Allow /app/
Disallow /*.integrator-includes.txt
Disallow /*.integrator-includes.json
Disallow /*.forhandlere-bilverksted.*/

msnbot

Rule Path
Allow /app/
Disallow /*.integrator-includes.txt
Disallow /*.integrator-includes.json
Disallow /*.forhandlere-bilverksted.*/

*

Rule Path
Disallow /*notification
Disallow /*.integrator-includes.txt
Disallow /*.integrator-includes.json
Disallow /*.forhandlere-bilverksted.*/
Back to top

Other Records

Field Value
sitemap https://www.volkswagen.no/no.sitemap.xml
Back to top

Comments

  • whitelist user agents / pages to visit for app-paths
  • DENY ACCESS FOR SPECIFIC BOT
  • User.Agent: [USER-Agent-Name]
  • Disallow: /
  • SITEMAP
Back to top