volkswagen-nutzfahrzeuge.de
robots.txt

Robots Exclusion Standard data for volkswagen-nutzfahrzeuge.de

Resource Scan

Scan Details

Site Domain volkswagen-nutzfahrzeuge.de
Base Domain volkswagen-nutzfahrzeuge.de
Scan Status Ok
Last Scan2025-03-16T06:48:29+00:00
Next Scan 2025-04-15T06:48:29+00:00

Last Scan

Scanned2025-03-16T06:48:29+00:00
URL https://volkswagen-nutzfahrzeuge.de/robots.txt
Redirect https://www.volkswagen-nutzfahrzeuge.de:443/robots.txt
Redirect Domain www.volkswagen-nutzfahrzeuge.de
Redirect Base volkswagen-nutzfahrzeuge.de
Domain IPs 75.2.48.58
Redirect IPs 23.46.230.149, 23.46.230.158, 2600:1413:b000:13::b857:c184, 2600:1413:b000:13::b857:c198
Response IP 23.49.60.51
Found Yes
Hash 894511888d74a120500b86e2a98b7764b57c3845ccf3b7e255190ee0fb2599eb
SimHash 652092d5c5d5

Groups

googlebot

Rule Path
Allow /app/
Disallow /*.integrator-includes.txt
Disallow /*.integrator-includes.json

slurp

Rule Path
Allow /app/
Disallow /*.integrator-includes.txt
Disallow /*.integrator-includes.json

bingbot

Rule Path
Allow /app/
Disallow /*.integrator-includes.txt
Disallow /*.integrator-includes.json

msnbot

Rule Path
Allow /app/
Disallow /*.integrator-includes.txt
Disallow /*.integrator-includes.json

*

Rule Path
Disallow /app/
Disallow /*notification
Disallow /app/konfigurator/vwn-de-vr/de*
Disallow /*.integrator-includes.txt
Disallow /*.integrator-includes.json
Disallow /de/modelle/schnell-verfuegbare-neu-jahres-und-gebrauchtwagen-suche.html/__app/search/car/*

Other Records

Field Value
sitemap https://www.volkswagen-nutzfahrzeuge.de/de.sitemap.xml

Comments

  • whitelist user agents / pages to visit for app-paths
  • default block
  • sitemap