bildungsklick.de
robots.txt
Robots Exclusion Standard data for bildungsklick.de
Resource Scan
Scan Details
| Site Domain | bildungsklick.de |
| Base Domain | bildungsklick.de |
| Scan Status | Ok |
| Last Scan | 2024-06-07T21:15:34+00:00 |
| Next Scan | 2024-07-07T21:15:34+00:00 |
Last Scan
| Scanned | 2024-06-07T21:15:34+00:00 |
| URL | https://bildungsklick.de/robots.txt |
| Domain IPs | 88.99.101.177 |
| Response IP | 88.99.101.177 |
| Found | Yes |
| Hash | 19a1c6a18dad2077b50db5d0868ba930b2ae186d7e2d87e2fca68c5fdcaac7d6 |
| SimHash | 7d0462780a96 |
Groups
*
| Rule | Path | Comment |
|---|---|---|
| Allow | / | - |
| Disallow | /typo3/ | - |
| Disallow | /typo3conf/ | - |
| Allow | /typo3conf/ext/ | - |
| Allow | /typo3temp/ | - |
| Disallow | /*?id=* | non speaking URLs |
| Disallow | /*cHash | no cHash |
| Disallow | /*tx_powermail_pi1 | no powermail thanks pages |
| Disallow | /*tx_form_formframework | no forms |
nutch
jyxobot
fast-webcrawler
fast enterprise crawler
biglotron
convera
gigabot
gigablast
exabot
gingercrawler
webmon
grub.org
usinenouvellecrawler
antibot
netresearchserver
speedy
fluffy
findlink
msrbot
panscient
yacybot
aisearchbot
ips-agent
tagoobot
mj12bot
woriobot
yanga
buzzbot
mlbot
yandexbot
yandeximages
yandexaccessibilitybot
yandexmobilebot
purebot
cyberpatrol
voilabot
baiduspider
citeseerxbot
spbot
twengabot
postrank
turnitinbot
scribdbot
page2rss
sitebot
linkdex
adidxbot
ezooms
dotbot
mail.ru_bot
discobot
heritrix
findthatfile
europarchive.org
nerdbynature.bot
ahrefs
fuelbot
crunchbot
indeedbot
mappydata
woobot
zoominfobot
privacyawarebot
multiviewbot
swimgbot
grobbot
eright
apercite
semanticbot
aboundex
domaincrawler
wbsearchbot
summify
ccbot
edisterbot
ec2linkfinder
gslfbot
aihitbot
intelium_bot
retrevopageanalyzer
lb-spider
sogou
careerbot
wotbox
wocbot
ichiro
lssrocketcrawler
drupact
webcompanycrawler
acoonbot
openindexspider
gnam gnam spider
web-archive-net.com.bot
coccoc
integromedb
content crawler spider
toplistbot
it2media-domain-crawler
ip-web-crawler.com
siteexplorer.info
elisabot
proximic
changedetection
arabot
wesee:search
niki-bot
crystalsemanticsbot
rogerbot
psbot
interfaxscanbot
cc metadata scaper
g00g1e.net
grapeshotcrawler
urlappendbot
brainobot
fr-crawler
binlar
simplecrawler
cxensebot
smtbot
bnf.fr_bot
a6-indexer
admantx
orangebot
memorybot
advbot
megaindex
semanticscholarbot
ltx71
nerdybot
xovibot
bubing
qwantify
tweetmemebot
crawler4j
findxbot
semrushbot
yoozbot
lipperhey
y!j
domain re-animator bot
addthis
livelap[bb]ot
capsulechecker
collection@infegy.com
istellabot
deusu
betabot
cliqzbot
mojeekbot
netestate ne crawler
safesearch microdata crawler
gluten free crawler
sonic
sysomos
trove
embedly
rankactivelinkbot
iskanie
safednsbot
veoozbot
slackbot
redditbot
datagnionbot
adbeat_bot
contxbot
electricmonk
garlikcrawler
vebidoobot
femtosearchbot
mindupbot
daum
pcore-http
moatbot
kosmiobot
pingdom
appinsights
phantomjs
gowikibot
piplbot
jetslide
newsharecounts
james bot
bark[rr]owler
tineye
socialrankiobot
trendictionbot
ocarinabot
epicbot
primalbot
gnowitnewsbot
leikibot
yak
paperlibot
digg deeper
dcrawl
snacktory
anderspinkbot
fyrebot
everyonesocialbot
mediatoolkitbot
luminator-robots
extlinksbot
ning
okhttp
nuzzel
omgili
pocketparser
yisouspider
um-ln
toutiaospider
muckrack
jamie's spider
ahc
netcraftsurveyagent
laserlikebot
jetty
upflow
thinklab
traackr.com
twurly
mastodon
http_get
dnyzbot
botify
behloolbot
brandverity
check_http
bdcbot
zumbot
ezid
icc-crawler
filterdb.iss.netcrawler
blp_bbot
bomborabot
buck
companybook-crawler
genieo
magpie-crawler
meltwaternews
moreover
newspaper
scoutjet
storygizebot
uptimerobot
outclicksbot
seoscanners
hatena
mauibot
alphabot
sbl-bot
ias crawler
adscanner
netvibes
acapbot
baidu-yunguance
bitlybot
blogmurabot
bot.araturka.com
bot-pge.chlooe.com
boxcarbot
btwebclient
contextad bot
digincore bot
disqus
fetch
fever
flamingo_searchengine
flipboardproxy
g2reader-bot
g2 web services
imrbot
k7mlwcbot
kemvibot
landau-media-spider
linkapediabot
vkshare
siteimprove.com
blexbot
dareboost
zuperlistbot
miniflux
feedspot
diffbot
tracemyfile
nimbostratus-bot
zgrab
pr-cy.ru
adstxtcrawler
datafeedwatch
zabbix
tangibleebot
axios
pulsepoint
cloudflare-alwaysonline
wordupinfosearch
webdatastats
zoombot
velenpublicwebcrawler
moodlebot
jpg-newsbot
outbrain
validator.nu
blackboard
icbot
bazqux
twingly
rivva
experibot
awesomecrawler
dataprovider.com
grouphigh
theoldreader.com
anyevent
uptimebot.org
nmap scripting engine
clickagy
caliperbot
mbcrawler
online-webceo-bot
b2b bot
addsearchbot
headlesschrome
checkmarknetwork
www.uptime.com
streamline3bot
serpstatbot
mixnodecache
simplescraper
jooblebot
fedoraplanet
friendica
bytespider
datanyze
trendsmapresolver
tweetedtimes
ntentbot
gwene
simplepie
searchatlas
superfeedr
feedbot
ut-dorkbot
serendeputybot
eyeotabot
officestorebot
neticle crawler
surdotlybot
linkisbot
awariosmartbot
awariorssbot
freewebmonitoring sitechecker
aspiegelbot
zenback bot
sentibot
domains project
pandalytics
vkrobot
bidswitchbot
tigerbot
nixstatsbot
atom feed robot
curebot
pagepeeker
vigil
rssbot
startmebot
jobboersebot
seewithkids
ninja bot
cutbot
bublupbot
brandonbot
ridderbot
yandexmetrika
yandexturbo
yandeximageresizer
yandexvideoparser
taboolabot
dubbotbot
finditanswersbot
infoobot
refindbot
blogtrafficd.d+ feed-fetcher
cincraw
dragonbot
voluumdsp-content-bot
freshrss
bitbot
| Rule | Path |
|---|---|
| Disallow | / |
Warnings
- 6 invalid lines.
Comments