Log In Sign Up

spardeingeld.de
robots.txt

Robots Exclusion Standard data for spardeingeld.de

Resource Scan

Scan Details

Site Domain spardeingeld.de
Base Domain spardeingeld.de
Scan Status Ok
Last Scan2025-09-26T19:28:29+00:00
Next Scan 2025-10-26T19:28:29+00:00

Last Scan

Scanned2025-09-26T19:28:29+00:00
URL https://spardeingeld.de/robots.txt
Domain IPs 2600:1901:0:dab1::, 34.54.31.19
Response IP 34.54.31.19
Found Yes
Hash c5549478082dd11bc572eae9837c3058ea7890db785d202d9e9cd11d1d7bffff
SimHash be1a53bd6fe4

Groups

*

Rule Path
Disallow /rd?
Disallow /rd2?
Disallow /color_bumper.xpml
Disallow /mkt.xpml*
Disallow /*_show--
Disallow /*_sort--
Disallow /classify
Disallow /*_start--
Disallow /*start%3D
Disallow /*show%3D
Disallow /*sort%3D
Disallow /*mkt_id%3D
Disallow /*_absoluteMaxPrice--
Disallow /*_absoluteMinPrice--
Disallow /*SEARCH_GO

googlebot

Rule Path
Disallow /rd?
Disallow /rd2?
Disallow /color_bumper.xpml
Disallow /mkt.xpml*
Disallow /*_show--
Disallow /*_sort--
Disallow /classify
Disallow /musik/
Disallow /newagemusik/
Disallow /komoedien/
Disallow /elektronischemusik/
Disallow /spokenword/
Disallow /jazzmusik_instrumental/
Disallow /cajunmusik/
Disallow /heavymetalmusik/
Disallow /tv_filmmusik/
Disallow /kindermusik/
Disallow /rock_popmusik/
Disallow /r_bhiphop_rapmusik/
Disallow /folkmusik/
Disallow /dvd_videofilme/
Disallow /abenteuerfilmeunddvds/
Disallow /westernfilmeunddvds/
Disallow /filmeunddvdsbesondererart/
Disallow /dramafilmeunddvds/
Disallow /horrorfilmeunddvds/
Disallow /kinderfilmeunddvds/
Disallow /komoedienfilmeunddvds/
Disallow /musicals/
Disallow /scififilmeunddvds/
Disallow /sportundfitnessfilmeunddvds/
Disallow /religioesefilmeunddvds/
Disallow /fremdsprachenfilmeunddvds/
Disallow /filmeunddvdsfuererwachsene/
Disallow /erotik_videosbuecher_mehr/
Disallow /buecher_zeitschriften/
Disallow /kinderbelletristik/
Disallow /lernbuecher/
Disallow /spokenword/
Disallow /philosophischeromane/
Disallow /erotischebuecher/
Disallow /geschichtsbuecher/
Disallow /zeitschriften_zeitungen/
Disallow /buero_schulzubehoer/
Disallow /damen_sonstigeaccessoires/
Disallow /diverses1/
Disallow /diversebuecher/
Disallow /diversefilmeunddvds/
Disallow /diversegeschenkeblumen_nahrungsmittel/
Disallow /diversegesundheit_pflege1/
Disallow /diversemusik/
Disallow /diversemoebel/
Disallow /diversesoftware/
Disallow /diversespeichergeraete/
Disallow /diversevideospiele/
Disallow /diverserbuerobedarf/
Disallow /diverses2/
Disallow /diversesbabyspielzeug/
Disallow /diversesbabyzubehoer/
Disallow /diversesbastelundkunstzubehoer/
Disallow /haus_gartendiverses/
Disallow /herren_sonstigeaccessoires/
Disallow /kinder_sonstigeaccessoires/
Disallow /sonstige
Disallow /weitereprodukte/
Disallow /weitererhaustierbedarf/
Disallow /*_start--
Disallow /*start%3D
Disallow /*show%3D
Disallow /*sort%3D
Disallow /*_absoluteMaxPrice--
Disallow /*_absoluteMinPrice--
Disallow /*SEARCH_GO

bingbot

Rule Path
Disallow /rd?
Disallow /rd2?
Disallow /color_bumper.xpml
Disallow /mkt.xpml*
Disallow /*_show--
Disallow /*_sort--
Disallow /classify
Disallow /musik/
Disallow /newagemusik/
Disallow /komoedien/
Disallow /elektronischemusik/
Disallow /spokenword/
Disallow /jazzmusik_instrumental/
Disallow /cajunmusik/
Disallow /heavymetalmusik/
Disallow /tv_filmmusik/
Disallow /kindermusik/
Disallow /rock_popmusik/
Disallow /r_bhiphop_rapmusik/
Disallow /folkmusik/
Disallow /dvd_videofilme/
Disallow /abenteuerfilmeunddvds/
Disallow /westernfilmeunddvds/
Disallow /filmeunddvdsbesondererart/
Disallow /dramafilmeunddvds/
Disallow /horrorfilmeunddvds/
Disallow /kinderfilmeunddvds/
Disallow /komoedienfilmeunddvds/
Disallow /musicals/
Disallow /scififilmeunddvds/
Disallow /sportundfitnessfilmeunddvds/
Disallow /religioesefilmeunddvds/
Disallow /fremdsprachenfilmeunddvds/
Disallow /filmeunddvdsfuererwachsene/
Disallow /erotik_videosbuecher_mehr/
Disallow /buecher_zeitschriften/
Disallow /kinderbelletristik/
Disallow /lernbuecher/
Disallow /spokenword/
Disallow /philosophischeromane/
Disallow /erotischebuecher/
Disallow /geschichtsbuecher/
Disallow /zeitschriften_zeitungen/
Disallow /buero_schulzubehoer/
Disallow /damen_sonstigeaccessoires/
Disallow /diverses1/
Disallow /diversebuecher/
Disallow /diversefilmeunddvds/
Disallow /diversegeschenkeblumen_nahrungsmittel/
Disallow /diversegesundheit_pflege1/
Disallow /diversemusik/
Disallow /diversemoebel/
Disallow /diversesoftware/
Disallow /diversespeichergeraete/
Disallow /diversevideospiele/
Disallow /diverserbuerobedarf/
Disallow /diverses2/
Disallow /diversesbabyspielzeug/
Disallow /diversesbabyzubehoer/
Disallow /diversesbastelundkunstzubehoer/
Disallow /haus_gartendiverses/
Disallow /herren_sonstigeaccessoires/
Disallow /kinder_sonstigeaccessoires/
Disallow /sonstige
Disallow /weitereprodukte/
Disallow /weitererhaustierbedarf/
Disallow /*_start--
Disallow /*start%3D
Disallow /*show%3D
Disallow /*sort%3D
Disallow /*_absoluteMaxPrice--
Disallow /*_absoluteMinPrice--
Disallow /*SEARCH_GO

slurp

Rule Path
Disallow /rd?
Disallow /rd2?
Disallow /color_bumper.xpml
Disallow /mkt.xpml*
Disallow /*_show--
Disallow /*_sort--
Disallow /classify
Disallow /musik/
Disallow /newagemusik/
Disallow /komoedien/
Disallow /elektronischemusik/
Disallow /spokenword/
Disallow /jazzmusik_instrumental/
Disallow /cajunmusik/
Disallow /heavymetalmusik/
Disallow /tv_filmmusik/
Disallow /kindermusik/
Disallow /rock_popmusik/
Disallow /r_bhiphop_rapmusik/
Disallow /folkmusik/
Disallow /dvd_videofilme/
Disallow /abenteuerfilmeunddvds/
Disallow /westernfilmeunddvds/
Disallow /filmeunddvdsbesondererart/
Disallow /dramafilmeunddvds/
Disallow /horrorfilmeunddvds/
Disallow /kinderfilmeunddvds/
Disallow /komoedienfilmeunddvds/
Disallow /musicals/
Disallow /scififilmeunddvds/
Disallow /sportundfitnessfilmeunddvds/
Disallow /religioesefilmeunddvds/
Disallow /fremdsprachenfilmeunddvds/
Disallow /filmeunddvdsfuererwachsene/
Disallow /erotik_videosbuecher_mehr/
Disallow /buecher_zeitschriften/
Disallow /kinderbelletristik/
Disallow /lernbuecher/
Disallow /spokenword/
Disallow /philosophischeromane/
Disallow /erotischebuecher/
Disallow /geschichtsbuecher/
Disallow /zeitschriften_zeitungen/
Disallow /buero_schulzubehoer/
Disallow /damen_sonstigeaccessoires/
Disallow /diverses1/
Disallow /diversebuecher/
Disallow /diversefilmeunddvds/
Disallow /diversegeschenkeblumen_nahrungsmittel/
Disallow /diversegesundheit_pflege1/
Disallow /diversemusik/
Disallow /diversemoebel/
Disallow /diversesoftware/
Disallow /diversespeichergeraete/
Disallow /diversevideospiele/
Disallow /diverserbuerobedarf/
Disallow /diverses2/
Disallow /diversesbabyspielzeug/
Disallow /diversesbabyzubehoer/
Disallow /diversesbastelundkunstzubehoer/
Disallow /haus_gartendiverses/
Disallow /herren_sonstigeaccessoires/
Disallow /kinder_sonstigeaccessoires/
Disallow /sonstige
Disallow /weitereprodukte/
Disallow /weitererhaustierbedarf/
Disallow /*_start--
Disallow /*start%3D
Disallow /*show%3D
Disallow /*sort%3D
Disallow /*_absoluteMaxPrice--
Disallow /*_absoluteMinPrice--
Disallow /*SEARCH_GO

adsbot-google

Rule Path
Disallow /rd2?
Disallow /rd?
Disallow /color_bumper.xpml
Disallow /*_start--
Disallow /*start%3D
Disallow /*show%3D
Disallow /*sort%3D

yahooseeker

Rule Path
Disallow /

becomebot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

cherrypicker

Rule Path
Disallow /

crescent internet toolpak

Rule Path
Disallow /

disco pump

Rule Path
Disallow /

emailsiphon

Rule Path
Disallow /

htdig

Rule Path
Disallow /

httrack

Rule Path
Disallow /

lnspiderguy

Rule Path
Disallow /

netprospector

Rule Path
Disallow /

perman surfer

Rule Path
Disallow /

psbot

Rule Path
Disallow /

szukacz

Rule Path
Disallow /

rpt-httpclient

Rule Path
Disallow /

teleport

Rule Path
Disallow /

telesoft

Rule Path
Disallow /

ultraseek

Rule Path
Disallow /

vspider

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

webemailextractor

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

webster pro

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

ubicrawler

Rule Path
Disallow /

doc

Rule Path
Disallow /

zao

Rule Path
Disallow /

internetseer.com

Rule Path
Disallow /

sitecheck.internetseer.com

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

fetch

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

teleport

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

webzip

Rule Path
Disallow /

linko

Rule Path
Disallow /

httrack

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

xenu

Rule Path
Disallow /

larbin

Rule Path
Disallow /

libwww

Rule Path
Disallow /

zyborg

Rule Path
Disallow /

download ninja

Rule Path
Disallow /

wget

Rule Path
Disallow /

grub-client

Rule Path
Disallow /

grub

Rule Path
Disallow /

k2spider

Rule Path
Disallow /

npbot

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

xovibot

Rule Path
Disallow /
Back to top

Comments

  • Add explict user-agent for AdsBot-Google as it ignores blanket includes
  • http://adwords.google.com/support/aw/bin/answer.py?hl=en&answer=38197
  • Crawlers that are kind enough to obey, but which we'd rather not have.
  • A growing list of pest bots
  • Some bots are known to be trouble, particularly those designed to copy
  • entire sites. Please obey robots.txt.
  • Sorry, wget in its recursive mode is a frequent problem.
  • Please refer to the manual and use it properly; there is a
  • wait = x number of seconds option you can use to set the delay between hits,
  • for instance.
  • The 'grub' distributed client has been *very* poorly behaved.
  • Doesn't follow robots.txt
  • Hits many times per second, not acceptable
  • http://www.nameprotect.com/botinfo.html
  • A capture bot, downloads gazillions of pages.
  • http://www.webreaper.net/
  • http://www.xovibot.net/
Back to top