Log In Sign Up

foulkes.de
robots.txt

Robots Exclusion Standard data for foulkes.de

Resource Scan

Scan Details

Site Domain foulkes.de
Base Domain foulkes.de
Scan Status Ok
Last Scan2024-09-15T06:55:34+00:00
Next Scan 2024-10-15T06:55:34+00:00

Last Scan

Scanned2024-09-15T06:55:34+00:00
URL https://www.foulkes.de/robots.txt
Domain IPs 87.106.14.51
Response IP 87.106.14.51
Found Yes
Hash 46d8998df1e26f562e09970109b9bc5c92a2fe09f9fab28666e44e00c2c762ec
SimHash 20b8177a5e82

Groups

*

Rule Path
Disallow /cgi-bin/
Disallow /css_js/
Disallow /inc/
Disallow /intern/
Disallow /php/
Disallow /templates/

googlebot

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

mediapartners-google

Rule Path
Allow /

adsbot-google

Rule Path
Allow /

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 300

baiduspider

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

megaindex.ru/2.0

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

majesticseo

Rule Path
Disallow /

backlinkcrawler

Rule Path
Disallow /

xovi

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

searchmetricsbot

Rule Path
Disallow /

search17

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

slysearch

Rule Path
Disallow /

findlinks

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

pixray-seeker

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

lb-spider

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

psbot

Rule Path
Disallow /

huaweisymantecspider

Rule Path
Disallow /

ec2linkfinder

Rule Path
Disallow /

htdig

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

discobot

Rule Path
Disallow /

linkdex.com

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

edisterbot

Rule Path
Disallow /

swebot

Rule Path
Disallow /

picmole

Rule Path
Disallow /

yeti

Rule Path
Disallow /

yeti-mobile

Rule Path
Disallow /

pagepeeker

Rule Path
Disallow /

catchbot

Rule Path
Disallow /

yacybot

Rule Path
Disallow /

netestatenecrawler

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

comodosslchecker

Rule Path
Disallow /

comodo-certificates-spider

Rule Path
Disallow /

gonzo

Rule Path
Disallow /

schrein

Rule Path
Disallow /

afiliaswebminingtool

Rule Path
Disallow /

suggybot

Rule Path
Disallow /

bdbrandprotect

Rule Path
Disallow /

bpimagewalker

Rule Path
Disallow /

updownerbot

Rule Path
Disallow /

lex

Rule Path
Disallow /

contentcrawler

Rule Path
Disallow /

dcpbot

Rule Path
Disallow /

kaloogabot

Rule Path
Disallow /

mlbot

Rule Path
Disallow /

icjobs

Rule Path
Disallow /

obot

Rule Path
Disallow /

webmastercoffee

Rule Path
Disallow /

qualidator

Rule Path
Disallow /

webinator

Rule Path
Disallow /

scooter

Rule Path
Disallow /

thunderstone

Rule Path
Disallow /

larbin

Rule Path
Disallow /

opidoobot

Rule Path
Disallow /

ips-agent

Rule Path
Disallow /

tineye

Rule Path
Disallow /

unisterbot

Rule Path
Disallow /

unister

Rule Path
Disallow /

reverseget

Rule Path
Disallow /

dotbot

Rule Path
Disallow /
Back to top

Comments

  • Google direkt ansprechen
  • Crawl Geschwindigkeit ausbremsen
  • Unnötige Bots sperren
  • Hello machines !
  • The webmaster for this site is:
  • <name>www.media-x-vision.de</name>
  • INFO: Wenn bots nicht reagieren .htaccess anpassen
  • RewriteCond %{HTTP_USER_AGENT} ^SEOkicks [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^SEOkicks-Robot [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^sistrix [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^MajesticSEO [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^BacklinkCrawler [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^xovi [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^XoviBot [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^MJ12bot [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^spbot [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^SearchmetricsBot [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^search17 [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^AhrefsBot [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^ia_archiver [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^TurnitinBot [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^SlySearch [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^findlinks [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^magpie-crawler [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^Pixray-Seeker [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^008 [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^Ezooms [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^lb-spider [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^WBSearchBot [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^psbot [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^HuaweiSymantecSpider [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^EC2LinkFinder [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^htdig [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^SemrushBot [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^discobot [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^linkdex.com [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^SeznamBot [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^EdisterBot [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^SWEBot [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^picmole [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^Yeti [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^Yeti-Mobile [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^PagePeeker [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^CatchBot [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^yacybot [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^netEstateNECrawler [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^SurveyBot [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^COMODOSSLChecker [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^Comodo-Certificates-Spider [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^gonzo [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^schrein [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^AfiliasWebMiningTool [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^suggybot [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^bdbrandprotect [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^BPImageWalker [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^Updownerbot [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^lex [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^ContentCrawler [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^DCPbot [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^KaloogaBot [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^MLBot [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^iCjobs [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^oBot [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^WebmasterCoffee [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^Qualidator [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^Webinator [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^Scooter [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^thunderstone [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^larbin [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^OpidooBOT [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^ips-agent [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^TinEye [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^UnisterBot [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^Unister [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^ReverseGet [NC]
  • RewriteRule ^.* - [F,L]
  • RewriteCond %{HTTP_USER_AGENT} ^DotBot [NC]
  • RewriteRule ^.* - [F,L]
Back to top

Warnings

  • 2 invalid lines.
Back to top