Log In Sign Up

charterboat24.com
robots.txt

Robots Exclusion Standard data for charterboat24.com

Resource Scan

Scan Details

Site Domain charterboat24.com
Base Domain charterboat24.com
Scan Status Ok
Last Scan2024-09-21T04:34:41+00:00
Next Scan 2024-10-21T04:34:41+00:00

Last Scan

Scanned2024-09-21T04:34:41+00:00
URL https://charterboat24.com/robots.txt
Redirect https://www.charterboat24.com/robots.txt
Redirect Domain www.charterboat24.com
Redirect Base charterboat24.com
Domain IPs 212.46.102.79
Redirect IPs 212.46.102.79
Response IP 212.46.102.79
Found Yes
Hash fe1e670fc5f7eaf9afca50a7894dcac80234156f9b8d5598c16b3dc62617c8de
SimHash aa3813aaeebb

Groups

*

Rule Path
Disallow /act/bsz.do
Disallow /act/bsl.do
Disallow /act/bsg.do
Disallow /act/bsk.do
Disallow /act/bst.do
Disallow /act/utk.do
Disallow /act/bal.do

seokicks

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

majesticseo

Rule Path
Disallow /

backlinkcrawler

Rule Path
Disallow /

xovi

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

searchmetricsbot

Rule Path
Disallow /

search17

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

slysearch

Rule Path
Disallow /

findlinks

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

pixray-seeker

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

lb-spider

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

psbot

Rule Path
Disallow /

huaweisymantecspider

Rule Path
Disallow /

ec2linkfinder

Rule Path
Disallow /

htdig

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

discobot

Rule Path
Disallow /

linkdex.com

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

edisterbot

Rule Path
Disallow /

swebot

Rule Path
Disallow /

picmole

Rule Path
Disallow /

yeti

Rule Path
Disallow /

yeti-mobile

Rule Path
Disallow /

pagepeeker

Rule Path
Disallow /

catchbot

Rule Path
Disallow /

yacybot

Rule Path
Disallow /

netestatenecrawler

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

comodosslchecker

Rule Path
Disallow /

comodo-certificates-spider

Rule Path
Disallow /

gonzo

Rule Path
Disallow /

schrein

Rule Path
Disallow /

afiliaswebminingtool

Rule Path
Disallow /

suggybot

Rule Path
Disallow /

bdbrandprotect

Rule Path
Disallow /

bpimagewalker

Rule Path
Disallow /

updownerbot

Rule Path
Disallow /

lex

Rule Path
Disallow /

contentcrawler

Rule Path
Disallow /

dcpbot

Rule Path
Disallow /

kaloogabot

Rule Path
Disallow /

mlbot

Rule Path
Disallow /

icjobs

Rule Path
Disallow /

obot

Rule Path
Disallow /

webmastercoffee

Rule Path
Disallow /

qualidator

Rule Path
Disallow /

webinator

Rule Path
Disallow /

scooter

Rule Path
Disallow /

thunderstone

Rule Path
Disallow /

larbin

Rule Path
Disallow /

opidoobot

Rule Path
Disallow /

ips-agent

Rule Path
Disallow /

tineye

Rule Path
Disallow /

unisterbot

Rule Path
Disallow /

unister

Rule Path
Disallow /

reverseget

Rule Path
Disallow /

dotbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap http://www.charterboat24.com/sitemap_index.xml
Back to top

Comments

  • alles erlaubt 21.7.04
  • 24.10.07 Merkzettel sperren
  • s.a. bewegungsDaten.setDeleteBwmzkBwmzdNichtZaehlbar
  • 5.12.07 BSL und BSD sperren
  • da nun nur noch die neuen statischen Seiten besucht werden sollen
  • 20.05.08
  • nach Regelung von jsessionID-Verhalten und flaechendeckender Einfuehrung
  • von nofollow weitere Seiten sperren
  • _Disallow: /act/bsd.do_
  • 20.12.07
  • Folgende Seiten werden durch rel='nofollow' in den Links
  • weitestgehend vor SuMa-Besuchen geschuetzt.
  • Inhalte sind nicht SuMa-relevant und
  • Links (Vererbung!) sollen nicht 'verschwendet' werden.
  • BSG, BSP, BSK, BST ,NWL, UTK, UTR
  • 3.5.2016
  • Aussperrung von Bots
  • User-agent: sistrix
  • Disallow: /
Back to top

Warnings

  • 2 invalid lines.
Back to top