automativ.de
robots.txt

Robots Exclusion Standard data for automativ.de

Resource Scan

Scan Details

Site Domain automativ.de
Base Domain automativ.de
Scan Status Ok
Last Scan2025-09-02T18:44:05+00:00
Next Scan 2025-09-09T18:44:05+00:00

Last Scan

Scanned2025-09-02T18:44:05+00:00
URL https://automativ.de/robots.txt
Domain IPs 85.214.100.198
Response IP 85.214.100.198
Found Yes
Hash 19d4310c8b98ffdc16846057d9d9cad8515a2c2c50cbf706a1fa33819124088e
SimHash a918180087b2

Groups

*

Rule Path
Disallow /admin/
Disallow /temp/

webreaper
webcopier
offline explorer
httrack
microsoft.url.control
emailcollector
penthesilea

Rule Path
Disallow /

Other Records

Field Value
sitemap http://www.automativ.de/sitemap_index.xml

Comments

  • ===================================
  • Folgende Seiten sollen nicht indexiert werden:
  • ===================================
  • ===================================
  • Schließe folgende Spider komplett aus:
  • ===================================