dokumente-online.com
robots.txt

Robots Exclusion Standard data for dokumente-online.com

Resource Scan

Scan Details

Site Domain dokumente-online.com
Base Domain dokumente-online.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-11-14T22:44:06+00:00
Next Scan 2026-02-12T22:44:06+00:00

Last Successful Scan

Scanned2024-06-30T07:03:09+00:00
URL https://dokumente-online.com/robots.txt
Domain IPs 104.21.61.118, 172.67.210.68, 2606:4700:3034::6815:3d76, 2606:4700:3036::ac43:d244
Response IP 172.67.210.68
Found Yes
Hash c6dffbe277cbe6cf9f706e313820f1bc631d483a95acf61a609870df75686ea3
SimHash b5701892e732

Groups

*

Rule Path
Disallow /checkout.php
Disallow /download_file.php
Disallow /download_unregistered3.php
Disallow /download_unregistered4.php
Disallow /userdaten-einsehen.php
Disallow /update_dokument.php
Disallow /select_pass.php
Disallow /basket.php
Disallow /outgoing_links_pruefen.php
Disallow /klick_auf_doc_ausloesen_aus_doc.php
Disallow /switch.php
Disallow /getprice.php
Disallow /download_bestaetigen.php
Disallow /paypal/process_neu.php
Disallow /paypal_checkout.php
Disallow /minipay_checkout.php
Disallow /micropayment_checkout.php
Disallow /statistic_1.php
Disallow /fo.php
Disallow /fono.php
Disallow /admin/
Disallow /temp/
Disallow /cdn-cgi/
Disallow /leckerli/

webreaper
webcopier
offline explorer
httrack
microsoft.url.control
emailcollector
penthesilea

Rule Path
Disallow /

Comments

  • ===================================
  • Folgende Seiten sollen nicht indexiert werden:
  • ===================================
  • ===================================
  • Schließe folgende Spider komplett aus:
  • ===================================