gtrkamur.ru
robots.txt

Groups

googlebot

*

yandex

yandexnews

yandexturbo

mozilla/5.0 (compatible; linux x86_64; zen_bot/2.0; +https://help.mail.ru/webmaster/indexing/robots)

gptbot
abonti
aboundex
aboundexbot
acunetix
admantx
adscanner
adstxtcrawlertp
afd-verbotsverfahren
ahrefsbot
aibot
aihitbot
aipbot
alexibot
alittleclient
alligator
allsubmitter
alphabot
anarchie
anarchy
anarchy99
ankit
anthill
apexoo
archive.org_bot
arquivo-web-crawler
arquivo.pt
aspiegel
aspseek
asterias
atomseobot
attach
autoemailspider
awariorssbot
awariosmartbot
backdoorbot
backlink-ceck
backlink-check
backlinkcrawler
backstreet
backweb
badass
bandit
barkrowler
batchftp
battleztarbazinga
bbbike
bdcbot
bdfetch
betabot
bigfoot
bitacle
blackhole
blackboard
blackwidow
blexbot
blow
blowfish
boardreader
bolt
botalot
brandprotect
brandwatch
buck
buddy
builtbottough
builtwith
bullseye
bunnyslippers
buzzsumo
bytespider
cah.io.community
calculon
catexplorador
cazoodlebot
ccbot
cegbfeieh
censyslnspect
checkl.exe
cheesebot
cherrypicker
cheteam
chinaclaw
chlooe
citoid
claritybot
clark-crawler
cliqzbot
cloudmapping
coccocbot
cocolyzebot
code87
cogentbot
cognitiveseo
collector
com.plumanalytics
copier
copyrightcheck
copyscape
cosmos
craflbot
gravityscan
crawl.sogou.com
crawler.feedback
crawler4j
crawlingathomeproject
crazywebcrawler
crescent
crunchbot
cshttp
curious
custo
cyotekwebcopy
databasedrivermysqli
datachaos
dataforseo.com
dataforseobot
dblbot
demandbase-bot
demon
deusu
devil
digincore
digitalpebble
dhbot
dirbuster
disco
discobot
discoverybot
dispatch
dittospyder
dnbcrawler-analytics
dnyzbot
domainappender
domaincrawler
domainsproject
domainsigmacrawler
domainsproject.org
domainstatsbot
domcopbot
dotbot
downloadwonder
dragonfly
drip
dsearch
dtsagent
easydl
ebingbong
ecatch
eccp
ecxi
eirgrabber
emailsiphon
emailwolf
erocrawler
evc-batch
evil
exabot
expresswebpictures
extlinksbot
extractor
extractorpro
extremepicturefinder
eyenetle
ezooms
facebookscraper
fdm
femtosearchbot
fhscan
fimap
firefox
flashget
flunky
foobot
freeuploader
frontpage
fuzz
fyberspider
fyrebot
g-i-g-a-b-o-t
galaxybot
genieo
germcrawler
getintent
getright
getweb
gigabot
goizilla
go-ahead-got-it
gopher
gotit
grabber
grabnet
grafula
grapefx
grapeshotcrawler
gridbot
gt::www
haansoft
haosouspider
harvest
havij
fffiadmasterseo
heritrix
heritrix
hloader
hmview
honolulubot
htmlparser
http::lite
httrack
humanlinks
hybridbot
iblog
id-search
idbot
idbte4m
ilsebot
imagefetch
imagesucker
indeedbot
indylibrary
infonavirobot
infotekies
instabid
intelliseek
interget
internetninja
intemetseer
internetvistamonitor
ips-agent
iria
irlbot
isitwp.com
iskanie
istellabot
iubenda-radar
jamesbot
jbrofiizz
jennyð’ot
jetcar
jetty
jikespider
jocwebspider
jorgee
justview
jyxobot
kenjinspider
keybottranslation-search-machine
keyworddensity
kinza
kozmosbot
lanshanbot
larbin
leap
leechftp
leechget
lexibot
lftp
libweb
libwhisker
liebaofast
lightspeedsystems
likse
linkbot
linkdexbot
linkextractorpro
linkpadbot
linkscan
linksmanager
linkwalker
linqiametadatadownloaderbot
linqiarssbot
linqiascrapebot
lipperhey
lipperheyspider
litemage_walker
lmspider
lnspiderguy
ltx71
lwp-request
lwp-trivial
lwp::simple
mag-net
magnet
magpie-crawler
majesticseo
majestic-seo
majestic12
markmonitor
markwatch
massdownloader
masscan
matahari
mauibot
mb2345browser
meanpathbot
meanpathbot
meanpathbot
mediatoolkitbot
mediawords
megalndex.ru
metauri
mfc_tear_sample
micromessenger
microsofturlcontrol
midowntool
miixpc
minefield
misterpix
mj12bot
mojeek
mojolicious
molokaibot
morfeusfuckingscanner
mr.4x3
msfrontpage
msiecrawler
msrabot
mtrobot
muhstik-scan
musobot
navroad
nearsite
needle
nessus
netvampire
netants
netcraft
netestatenecrawler
netlyzer
netmechanic
netspider
nettrack
netvibes
netzip
nextgensearchbot
nibbler
nicerspro
niki-bot
nikto
nimblecrawler
nimbostratus
ninja
nmap
npbot
nuclei
nutch
obot
octopus
offlineexplorer
offlinenavigator
oncrawl
openai
openai.com
openfmd
openlinkprofiler
openvas
openvas
orangebot
orangespider
outclicksbot
outfoxbot
pageanalyzer
pagescorer
pageanalyzer
pagegrabber
pagescorer
pagething.com
pandalytics
panscient
papafoto
pavuk
pcbrowser
pecl::http
peoplepal
petalbot
phpcrawl
pi-monster
picscout
picsearch
picturefinder
piepmatz
pimonster
pixray
pleasecrawl
plumanalytics
pockey
poe-component-client-http
polarisversion
probe-image-size
probethenet
propowerbot
prowebwalker
proximic
psbot
pump
pu_in
pxbroker
pycurl
querynmetasearch
quick-crawler
rankactive
rankactivelinkbot
rankflex
rankingbot
rankingbot2
rankivabot
rankurbot
re-re
realdownload
reaper
rebelmouse
recorder
redesscrapy
reget
repomonkey
ripper
ripz
rocketcrawler
rogerbot
rssingbot
slz.ru
saleslntelligent
satoristudio.net
sbider
scalaj-http
scan.lol
scanalert
scanbot
scoutjet
scrapy
screaming
screenerbot
screpybot
searchestate
searchmetricsbot
seekport
seekportbot
semanticjuice
semrush
semrushbot
sentibot
senutobot
seobility
seobilitybot
seocompany.store
seokicks
seokicks-robot
seolyticscrawler
seomoz
seoprofiler
seoscanners
seositecheckup
seostar
seostats
serpstatbot
sexsearcher
shodan
siphon
sistrix
sitesucker
sitebeam
sitechecker.pro
sitecheckerbotcrawler
siteexplorer
siteimprove
sitelockspider
siteripz
sitesnagger
sitesucker
sitevigil
slysearch
smartdownload
smtbot
snake
snapbot
snoopy
socialranklobot
sociscraper
sogouwebspider
sogouspider
sosospider
sottopop
spacebison
spammen
spankbot
spanner
spbot
spinn3r
sputnikbot
spyfu
sp_auditbot
sqlmap
sqlworm
sqworm
steeler
stripper
sucker
sucuri
superbot
superhttp
surfbot
surveybot
suzuran
swiftbot
sysscan
szukacz
tophackteam
t8abot
takeout
teleport
teleportpro
telesoft
telesphoreo
telesphorep
theintraformant
thenomad
thumbor
tighttwatbot
titan
toata
toweyabot
tracemyfile
trendiction
trendiction.com
trendiction.de
trendictionbot
truerobot
turingos
tumitin
tumitinbot
twengabot
twice
typhoeus
ubermetrics-technologies.com
unisterbot
upflow
urlywarning
urly.warning
v-bot
vacuum
vagabondo
vbproject
vci
velenpublicwebcrawler
vericitecrawler
vidiblescraper
virusdie
voideye
voil
voltron
voyagerx.com
wasalive-bot
wbsearchbot
webauto
webcollage
webenhancer
webfetch
webfuck
webpix
websauger
websucker
webalta
webauto
webbandit
webcollage
webcopier
webdav
webenhancer
webfetch
webfuck
webgains-bot
webgois
weblmagecollector
webleacher
webmasterworldforumbot
webmeup-crawler
webpix
webpros.com
webprosbot
webreaper
websauger
webshag
websitequester
websiteextractor
websitequester
webster
webstripper
websucker
webwhacker
webzip
wesee
whack
whacker
whatweb
who.isbot
widow
winhttrack
wiseguysrobot
wisenutbot
wonderbot
woobot
wotbox
wprecon
wpscan
www-collector-e
www-mechanize
www::mechanize
wwwoffle
xaldonwebspider
xaldonjwebspider
xenu
xpymepl.exe
youdaobot
zade
zauba
zauba.io
zermelo
zeus
zgrab
zitebot
zmeu
zoombot
zoominfobot
zumbot
zyborg
aolbuild
applebot
developers.facebook.com
docomo
facebookextemalhit
facebookplatform
linkedlnbot
slackbot
slackbot-linkexpanding
twitterbot

Other Records

Warnings

• 5 invalid lines.
• `clean-param` is not a known field.
• `host` is not a known field.