Log In Sign Up

rohkostwiki.de
robots.txt

Robots Exclusion Standard data for rohkostwiki.de

Resource Scan

Scan Details

Site Domain rohkostwiki.de
Base Domain rohkostwiki.de
Scan Status Ok
Last Scan2024-11-10T02:13:12+00:00
Next Scan 2024-12-10T02:13:12+00:00

Last Scan

Scanned2024-11-10T02:13:12+00:00
URL https://rohkostwiki.de/robots.txt
Redirect https://www.rohkostwiki.de/robots.txt
Redirect Domain www.rohkostwiki.de
Redirect Base rohkostwiki.de
Domain IPs 5.175.14.126
Redirect IPs 5.175.14.126
Response IP 5.175.14.126
Found Yes
Hash c0973fff1c5dddd98a7f9c47a329ce78743e6b24409efde331f35bcc1c576958
SimHash da489c884621

Groups

nutch

Rule Path
Disallow /

obot

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

backlinkcrawler

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

dubaiindex

Rule Path
Disallow /

crawler4j

Rule Path
Disallow /

coccoc

Rule Path
Disallow /

it2media-domain-crawler

Rule Path
Disallow /

smtbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

ltbot

Rule Path
Disallow /

gigablastopensource

Rule Path
Disallow /

bubing

Rule Path
Disallow /

lcc

Rule Path
Disallow /

yooz

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /

telegrambot

Rule Path
Disallow /

psbot

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

deepcrawl

Rule Path
Disallow /

feedly

Rule Path
Disallow /

spbot

Rule Path
Disallow /

iskanie

Rule Path
Disallow /

plukkie

Rule Path
Disallow /

deusu

Rule Path
Disallow /

domainappender

Rule Path
Disallow /

safednsbot

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

siteexplorer

Rule Path
Disallow /

wonderbot

Rule Path
Disallow /

uptimebot

Rule Path
Disallow /

ltx71

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

linguee

Rule Path
Disallow /

lexxebot

Rule Path
Disallow /

daum

Rule Path
Disallow /

domaincrawler

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

findxbot

Rule Path
Disallow /

heritrix

Rule Path
Disallow /

surdotlybot

Rule Path
Disallow /

tineye

Rule Path
Disallow /

bdcbot

Rule Path
Disallow /

idbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

adscanner

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

seekport

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

semantic health web crawler (shc-info.gecko.hs-heilbronn.de)

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

baiduspider-image
googlebot-image
msnbot-media

Rule Path
Disallow /w/index.php
Disallow /images/archive/

*

Rule Path
Disallow /w/index.php
Disallow /wiki/Spezial%3A
Disallow /wiki/spezial%3A
Disallow /wiki/Diskussion%3A
Disallow /wiki/diskussion%3A
Disallow /wiki/Rohkost-Wiki_Diskussion%3A
Disallow /wiki/rohkost-wiki_diskussion%3A
Disallow /wiki/Benutzer%3A
Disallow /wiki/Hilfe%3A
Disallow /wiki/hilfe%3A
Disallow /wiki/Rohkost-Wiki%3A
Disallow /wiki/rohkost-wiki%3A
Disallow /wiki/Test-Seite_f%C3%BCr_das_Schreiben_im_Wiki
Disallow /wiki/Testseite_die_zweite
Disallow /linkpruefer.html
Disallow /wiki/%241
Back to top

Comments

  • Erlaubte Zugriffe, einzelne Verzeichnisse gesperrt
  • Erlaubte Zugriffe, einzelne Seiten und Verzeichnisse gesperrt
  • Spezial- und Diskussionsseiten herausnehmen
  • Meta-Seiten herausnehmen und Datenschutz
  • Testseiten und Skript-Variable herausnehmen
Back to top