partner.comes.com.pl
robots.txt

Groups

*

adscanner
advbot
afilias web mining tool
ahrefsbot
aihitbot
alphabot
alphaseobot
archive.org_bot
backlinkcrawler
baiduspider
baidu
baiduspider+
baiduspider
barkrowler
bdbrandprotect
beambot
betabot
blexbot
bpimagewalker*
bubing
catchbot
ccbot
centurybot
checkmarknetwork
cipacrawler
cliqzbot
cms crawler
comodo ssl checker
comodo-certificates-spider
content crawler
crawler4j
curl
damn404 broken link checker
dataprovider
daum
dcpbot
deepcrawl
discobot
dispatch
dnyzbot
docomo/2.0
domaincheck.io
domaincrawler
domainsigmacrawler
domainsonocrawler
domainstatsbot
dotbot
drupact
ec2linkfinder
edisterbot
eright
exabot
exb language crawler
experibot
extlinksbot
ezooms
finbot
findlinks
garlik
gigablastopensource
gigabot
g-i-g-a-b-o-t
go 1.1 package http
gonzo
gridbot
gslfbot
htdig
httpget
huaweisymantecspider
hyper_python
ia_archiver
icjobs
idbot
infohelfer
ips-agent
ipsagent
it2media-domain-crawler
i4ds-bot
jagoona_bot
java
jitbit ssl checker
jobboerse
kaloogabot
komodiabot
larbin
lb-spider
lcc
lex
libwww-perl
lightspeedsystemscrawler
linguee bot
linkdexbot
linkpadbot
linkpad
linkstats
lssrocketcrawler
ltx71
magpie-crawler
mail.ru_bot
mauibot
meanpath
meanpathbot
megaindex
metajobbot
mindupbot
mixrankbot
mfibot
mia
mj12bot
mlbot
mojeekbot
msiecrawler
netcraft
netcraftsurveyagent
netestate ne crawler
nimbostratus-bot
nutch
obot
openindexspider
openlinkprofiler
opidoobot
pagepeeker
picmole
pixray-seeker
plukkie
pocketparser
pycurl
python
python-requests
psbot
publiclibraryarchive
qualidator*
qwantify
researchbot
researchscan
reverseget
riddler
safedns
safednsbot
savetheworldheritage
savetheworldheritage.org
schrein
scooter
scrapy
screaming frog
screenerbot
semanticbot
semantic health web crawler
semrushbot
semrushbot
semrushbot-sa
senderproof
seodiver
seokicks
seokicks-robot
searchmetricsbot
seoscanners
seznambot
sheerexplorer
sheerseo
sistrix
sistrix
siteexplorer
siteimprove
slysearch
smtbot
sogou web spider
solomonobot
solomonolinkchecker
sosospider
spbot
special_archiver
spiderlytics
spyonweb
ssearch
ssl labs
suggybot
surveybot
swebot
tineye
tracemyfile
trendictionbot
twitterbot
turnitinbot
unister*
unisterbot
updownerbot
uptimebot
vebidoobot
wappalyzer
waybackarchive.org
waybackarchive
wbsearchbot
webbandit
webinator
webmastercoffee
webmeup-crawler
webofantbot
wevikabot
wget
wlmhttptransport
woobot
wotbox
wotbox
www-mechanize
x28-job-bot
xovibot
xofi
yacybot
yak
yandexbot
yandex
yandeximages
yeti
yeti-mobile
yisouspider
youdaobot
zgrab
meta-externalagent
meta-externalfetcher/1.1
meta-externalfetcher
facebookexternalhit/1.1
facebookcatalog/1.0
facebookexternalhit
facebookcatalog
claudebot
backlinksextendedbot

Other Records

Comments

• Blocked bots

Warnings

• 3 invalid lines.