savolit.co.uk
robots.txt

Groups

admantx
ahc/
aibot
alittle client
amznkassocbot
aspseek
abonti
aboundex
aboundexbot
accserver
activeworlds
acunetix
adstxtcrawlertp
afd-verbotsverfahren
ahrefsbot
aihitbot
aipbot
alexibot
allsubmitter
alligator
alphabot
anarchie
anarchy
anarchy99
ankit
anthill
anyconnect
apache-httpclient
apache2449traversalplugin
apexoo
aplix
aqua_products
aspiegel
aspiegelbot
asterias
atomseobot
attach
awariorssbot
awariosmartbot
bbbike
bdcbot
bdfetch
bfac
bidubrowser
blexbot
bw/1.1
backdoorbot
backstreet
backweb
backlink-ceck
backlinkcrawler
badass
baidu
baiduspider
bandit
barkrowler
batchftp
battleztar bazinga
battleztarbazinga
beonex
betabot
bigfoot
bitacle
bizcodomainvalidationaudit
black hole
blackhole
blackwidow
blackboard
bleriot
bloodhound/nutch
blow
blowfish
boardreader
bolt
bookmark search tool
botalot
botrighthere
brandprotect
brandwatch
buck
buddy
builtbottough
builtwith
bullseye
bunnyslippers
buzzsumo
bytespider
catexplorador
ccbot
cispa vulnerability notification
cispa web analyser
cispa webcrawler
code87
cpython
cshttp
calculon
carbon
cazoodlebot
cegbfeieh
censysinspect
chameleon
changedetection
charon
chatgpt-user
cheteam
checkmarknetwork
cheesebot
cherrypicker
cherrypickerelite
cherrypickerse
chinaclaw
chlooe
citoid
claritybot
clickagy
clickbot
cliqzbot
cloud mapping
cloudwatchsynthetics
cloudinary
cloudmapping
cocolyzebot
cogentbot
collector
copernic
copier
copyrightcheck
copyscape
cosmos
craftbot
crawling at home project
crawlson
crazywebcrawler
crescent
crescent internet toolpak
crowdtanglebot
crunchbot
curious
custo
cyotekwebcopy
cyveillance
dblbot
df bot
diibot
dsearch
dts agent
dtsagent
darwin_i386
datacha0s
dataforseobot
databasedrivermysqli
datanyze
dataprovider
daum
demandbasepublisheranalyzer
demon
detectify
deusu
devil
digincore
digitalpebble
dingtalk
dirbuster
disco
discobot
discordbot
discoverybot
dispatch
dittospyder
dnbcrawler-analytics
dnyzbot
domcopbot
domainappender
domaincrawler
domainsigmacrawler
domainstatsbot
domains project
domainsproject
dotbot
download wonder
downloadwonder
dragonfly
drip
eccp/
eccp/1.0
elinks/
email siphon
email wolf
emailsiphon
easydl
ebingbong
ecxi
eirgrabber
emailcollector
emailwolf
embedly
erocrawler
evil
exabot
expanse
express webpictures
expresswebpictures
extlinksbot
extractor
extractorpro
extreme picture finder
extremepicturefinder
eyenetie
ezooms
f-secure radar
fdm
fhscan
facebookbot
fairad client
faraday
feedzcollector
femtosearchbot
fimap
firefox/7.0
flaming attackbot
flashget
flipboard
flunky
foobot
foregenix
freeuploader
frontpage
funwebproducts
fuzz
fuzz faster u fool
fyberspider
fyrebot
g-i-g-a-b-o-t
gptbot
grequests
gt::www
gaisbot
galaxybot
garlikcrawler
genieo
germcrawler
getright
getweb
getintent
gigablast
gigabot
go http package
go!zilla
go-ahead-got-it
go-http-client
gozilla
google-extended
gotit
gpcsupbot
grabnet
grabber
grafula
grammarly
grapefx
grapeshot
grapeshotcrawler
grapeshotcrawler/2.0
gridbot
grover
guzzlehttp
headmasterseo
hmview
htmlparser
http banner detection
http::lite
httpie
httrack
haansoft
haboubianis
hakai
haosouspider
harvest
havij
headless
headlesschrome
hello, world
heritrix
hitleap
hloader
honolulubot
httpurlconnection
huaweisymantecspider
hubspot
humanlinks
hybridbot
iab atq team adform robot
ibrowse
idbte4m
idbot
irlbot
iblog
id-search
iframely
ilsebot
image fetch
image sucker
imagefetch
imagesucker
imagevisu
indeedbot
indy library
indylibrary
inferno
infonavirobot
infopath
infotekies
innovantagebot
intelliseek
interget
internet ninja
internet security survey bot
internetninja
internetseer
ioncrawl
iria
iron33
iskanie
istellabot
joc web spider
jocwebspider
jamesbot
java/1.8.0
jbrofuzz
jennybot
jersey
jetcar
jetty
jikespider
joomla
jorgee
justview
jyxobot
ktxn
kenjin spider
kenjinspider
keybot translation-search-machine
keyword density
keyworddensity
kinza
komodiabot
kozmosbot
krebsonsecurity
lnspiderguy
lwp::simple
lanshanbot
larbin
leap
leechftp
leechget
lexibot
lftp
libvlc
libweb
libwhisker
liebaofast
liferea
lightspeedsystemscrawler
lightspeedsystems
likse
linespider
lingewoud-550-spyder
linkscan
linkwalker
linkbot
linkdexbot
linkextractorpro
linkpadbot
linksmanager
linqiametadatadownloaderbot
linqiarssbot
linqiascrapebot
linux gnu
lipperhey
lipperhey spider
lipperheyspider
litemage_walker
livelapbot
lkx-traversalhttpplugin
lmspider
ltx71
mfc_tear_sample
midown tool
midowntool
miixpc
mj12bot
mqbot
mqqbrowser
msfrontpage
msiecrawler
mtrobot
mag-net
magnet
mail.ru
mail.ru_bot
majestic seo
majestic-seo
majestic12
majesticseo
markmonitor
markwatch
mass downloader
massdownloader
masscan
mata hari
matahari
mauibot
maxpointcrawler
maxthon
mb2345browser
meanpath bot
meanpathbot
mediatoolkitbot
megaindex
megaindex.ru
meltwater
mermoz
metauri
methbot
micromessenger
microsoft data access
microsoft url control
microsoftdataaccess
microsofturlcontrol
minefield
mister pix
misterpix
mixnodecache
moblie safari
mojeek
mojeekbot
mojolicious
molokaibot
moobot
morfeus fucking scanner
morfeusfuckingscanner
mozilla/4.0 (compatible; bullseye; windows 95)
mozlila
mozzila
mr.4x3
msrabot
musobot
netcomplete
nicerspro
npbot
name intelligence
nameintelligence
nameprotect
naverbot
navroad
nearsite
needle
neevabot
nessus
net m@nager
net vampire
netants
netlyzer
netmechanic
netspider
netsystemsresearch
netvampire
netzip
netcraft
netcraft web server survey
netcraftsurveyagent
nettrack
netvibes
newsgatoronline
nextgensearchbot
nibbler
nicecrawler
niki-bot
nikto
nimblecrawler
nimbostratus
ninjbot
ninja
nmap
nuclei
nutch
odp link checker
oppo a33
oppoa33
octopus
offline explorer
offline navigator
offlineexplorer
offlinenavigator
omgili
omgilibot
oncrawl
open-source project
openlinkprofiler
openurl
openvas
openbot
openfind
openfind data gatherer
openvas
oracle ultra search
orangebot
orangespider
orbbot
orbot
outclicksbot
outfoxbot
owlin bot
pecl::http
phpcrawl
poe-component-client-http
psinet
page analyzer
pageanalyzer
pagegrabber
pagescorer
pagething.com
palo alto networks
pandalytics
panscient
papa foto
papafoto
pavuk
pe7kata
peoplepal
perman
petalbot
phantomjs
photon
pi-monster
picscout
picsearch
picturefinder
piepmatz
pimonster
piplbot
pixray
pleasecrawl
pockey
postmanruntime
propowerbot
prowebwalker
probethenet
project 25499
project-resonance
proximic
psbot
pu_in
pubmatic crawler bot
pump
pxbroker
pycurl
python
python-urllib
qqbrowser
qqgamehall
qtwebengine
queryn metasearch
querynmetasearch
queryseekerspider
quick-crawler
qwantify
rma
rss popper
rssingbot
radiation retriever
rankactive
rankactivelinkbot
rankflex
rankingbot
rankingbot2
rankivabot
rankurbot
re-re
reget
reactornetty
realdownload
reaper
rebelmouse
recon-ng
recorder
redesscrapy
repomonkey
restsharp
ripper
rocketcrawler
rogerbot
roku/
rome client
sbider
seokicks
seokicks-robot
seolyticscrawler
seoprofiler
seostats
sistrix
smtbot
safednsbot
salesintelligent
scamadviserexternalhit
scanalert
scanbot
scoutjet
scrapy
screaming
screaming frog
screenerbot
screpybot
searchestate
searchmetricsbot
securitybot
seekport
seekportbot
semanticjuice
semrush
semrushbot
sentibot
senutobot
seositecheckup
seobilitybot
seomoz
serendeputybot
seznam
seznambot
shodan
sidetrade
siphon
site sucker
sitecheckerbotcrawler
siteexplorer
sitelockspider
sitesnagger
sitesucker
sitebeam
siteimprove
sitevigil
slysearch
smartdownload
snake
snapbot
snoopy
socialrankiobot
sociscraper
sogou web spider
sogouwebspider
sosospider
sottopop
spacebison
spammen
spankbot
spanner
spbot
speedy spider
spinn3r
sproose
sputnikbot
sqlmap
sqlworm
sqworm
startpageprivateimageproxy
steeler
stormcrawler
stripper
sucker
sucuri
superbot
superhttp
surdotlybot
surfbot
surveybot
suzuran
swiftbot
sylera
symfony browserkit
szukacz
t0phackteam
t8abot
ttd-content
teleport
teleportpro
telesoft
telesphoreo
telesphorep
tennysoncrawler
test certificate info
testbot
the intraformant
the knowledge ai
theintraformant
thenomad
thumbor
tighttwatbot
tineye
titan
toata
toweyabot
tracemyfile
traversalhttpplugin
trendiction
trendictionbot
true_robot
turingos
turnitin
turnitinbot
twengabot
twice
twiceler
typhoeus
url control
url_spider_pro
urly warning
urly.warning
urlywarning
ubicrawler
unisterbot
upflow
urldispatcher
v-bot
vb project
vbproject
vci
vci webviewer
vlc/
vacuum
vagabondo
velenpublicwebcrawler
vericitecrawler
vidiblescraper
virusdie
voideye
voil
voltron
vsusearchspider
wasalive-bot
wbsearchbot
webdav
wisenutbot
wlmhttptransport
wpscan
www-collector-e
www-mechanize
www::mechanize
wwwoffle
wallpapers
wallpapers/3.0
wallpapershd
wappalyzer
wayback machine
wechat
wesee
web auto
web collage
web enhancer
web fetch
web fuck
web image collector
web pix
web sauger
web sucker
web-sniffer
webauto
webbandit
webcollage
webcopier
webenhancer
webfetch
webfuck
webgo is
webgois
webimagecollector
webleacher
webpix
webreaper
websauger
webstripper
websucker
webwhacker
webzip
webalta
webmasterworldforumbot
webshag
website quester
websiteextractor
websitequester
webster
webster pro
wellknownbot
wfuzz
wget
wget/
whack
whacker
whatweb
who.is bot
who.isbot
widow
wikido
winhttrack
winhttp
winhttpclient
winhttprequest
wiseguys robot
wiseguysrobot
wonderbot
woobot
wotbox
wprecon
xaldon webspider
xaldonwebspider
xaldon_webspider
xenu
yabrowser
yandex
yandexaccessibilitybot
yandexbot
yandeximages
yanga
yeti
yisouspider
youdaobot
zade
zauba
zermelo
zerofox
zeus
zeus 32297 webster pro
zeus link scout
zipcommander
zitebot
zmeu
zoombot
zoominfobot
zumbot
zyborg
absinthe
accountsd
adscanner
advanced email extractor
ahrefs
aiohttp
anasbousselham
arachni/
archive.org_bot
arquivo-web-crawler
arquivo.pt
autoemailspider
autogetcontent
axios
b2w/
backlink-check
bidswitchbot
bilbo
bit.ly/2w6px8s
bitlybot
bizopscompanies
bl.uk_lddc_bot
bl.uk_lddc_renderbot
bl.uk_ldfc_bot
black widow
blackwidow
bluefish 0.6 html editor
borchuk
brokenlinkcheck
brutus
brutus/aet
bsqlbf
cah.io.community
centuryb.o.t9
centurybot9
cgichk
check1.exe
cisco-torch
clark-crawler
cloudsystemnetworks
clshttp
coccocbot
cocrawler
cognitiveseo
colly
colly - https://github.com/gocolly/colly/v2
com.plumanalytics
commix
commoncrawl
core-project/1.0
crawl.sogou.com
crawler.feedback
crawler4j
crawler_eb_germany
crimscanner/
curl/7.19.7
curl/7.29.0
curl/7.47.0
curl/7.58.0
curl/7.61.1
curl/7.64.0
curl/7.68.0
curl/7.78.0
datacha0s
dataforseo.com
dataforseobot
dav.pm/v
demandbase-bot
dirbuster
dnsscan
domainsproject.org
domino hunter
dotdotpwn
ecatch
email extractor
entferbot
evc-batch
eventures
facebook_www_script
facebookscraper
fanpa.html
fasthttp
fhscan core 1.
floodgate
gbrmss
gdnplus
get-minimal
github5
gobuster
gootkit auto-rooter scanner
gopher
got (https://github.com/sindresorhus/got)
grabber
grapefx
grendel-scan
grub
grub-client
havij
heritrix
httplib
httpunit
httpx
ia_archiver
ias-ir
ias-or
ias-va
ichiro
idg/it
img2dataset
inspath
instabid
internal dummy connection
internet ninja
internetvista monitor
internetvistamonitor
iodc
ips-agent
isitwp
isitwp.com
iubenda-radar
jaascois
jimmysawczuk/recon
l9explore
l9tcpid
leakix
libidn
libssh2
libwww-perl
linkdexbot
linkfluence
lkxscan
ltx71
lua-resty-http
lwp-request
lwp-trivial
magpie-crawler
masscan
meanpathbot
mediawords
metis
microsoft.windowscommunicationsapps
mixnode
moatbot
moget
morfeus fucking scanner
mozilla/4.0 (compatible)
mozilla/4.0 (compatible; msie 6.0; win32)
mozilla/5.0 sf/
mozilla/5.0 sf//
muhstik-scan
mysqloit
n-stealth
naver.me
nessus
netestate ne crawler
netestatenecrawler
netsparker
newspaper
ngx_lua
nikto
nmap nse
nmap scripting engine
nmap-nse
node-fetch
nominet
nsauditor
obot
odysseus
okhttp
onlinepbx
openai
openai.com
openvas
owned-you
page scorer
paloaltonetworks
pangolin
paros
pathbuster
pcbrowser
plumanalytics
pmafind
pmoz.info
polaris version
polarisversion
probe-image-size
prog.customcrawler
projectdiscovery
prowebwalker
purely.domains
pymills-spider/
python-httplib2
python-requests
qualys was
r00ts3c-owned-you
rest-client
ripz
rogerbot
s.t.a.l.k.e.r.
s1z.ru
satoristudio
satoristudio.net
scalaj-http
scan.lol
scraper
searchpreview
security scan
seobility
seocompany.store
seoscanners
seostar
serpstatbot
sexsearcher
sindresorhus
sitechecker.pro
siteripz
sogouspider
sp_auditbot
spawning-ai
spaziodati
springenwerk
spyfu
sql power injector
sqlmap
sqlninja
startmebot
struts-pwn
symfony browserkit
sysscan
t3versions
takeout
tchelebi
teh forest lobster
this is an exploit
tocrawl
toata dragostea
toata dragostea mea pentru diavola
trendiction.com
trendiction.de
typhoeus
ubermetrics-technologies.com
udu/
uil2pn
ultimate_sitemap_parser
user-agent:
vega/
vinn.com
virustotalcloud
voideye
voyagerx.com
w3af.org
w3af.sf.net
w3af.sourceforge.net
webbandit
webgains-bot
webinspect
webmeup-crawler
webpros.com
webprosbot
webshag
webtrends security analyzer
webvulnscan
whatweb
whcc/
willnorris/imageproxy
winhttp.winhttprequest
woorankreview
wordpress hash grabber
wp_is_mobile
wwwster
x09mozilla
x22mozilla
xforce
xmlrpc exploit
xpymep1
xpymep1.exe
xtc botnet
xtcbot
zauba.io
zd4life
zerossl
zgrab
zh-cn
zh_cn
zhanzhang
zlib
zmeu

*

Other Records

Warnings

• 8 invalid lines.