matchthememory.com
robots.txt

Groups

*

No rules defined. All paths allowed.

Other Records

abonti
aboundex
aboundexbot
acunetix
admantx
adscanner
adstxtcrawlertp
afd-verbotsverfahren
ahrefsbot
aibot
aihitbot
aipbot
alexibot
alittle client
alligator
allsubmitter
alphabot
anarchie
anarchy
anarchy99
ankit
anthill
anthropic-ai
apexoo
aqua_products
arquivo-web-crawler
arquivo.pt
aspiegel
aspseek
asterias
asterias
atomseobot
attach
autoemailspider
awario.com
awariobot
awariorssbot
awariosmartbot
b2w/0.1
backdoorbot
backdoorbot/1.0
backlink-ceck
backlink-check
backlinkcrawler
backlinksextendedbot
backstreet
backweb
badass
bandit
barkrowler
batchftp
battleztar bazinga
bbbike
bdcbot
bdfetch
betabot
bigfoot
bitacle
black.hole
black hole
blackboard
blackwidow
blexbot
blow
blowfish
blowfish/1.0
boardreader
bolt
bookmark search tool
bot mailto:craftbot@yahoo.com
botalot
botrighthere
brandprotect
brandwatch
buck
buddy
builtbottough
builtwith
bullseye
bullseye/1.0
bunnyslippers
buzzsumo
bytespider
cah.io.community
calculon
catexplorador
cazoodlebot
ccbot
cegbfeieh
censysinspect
chatgpt-user
check1.exe
cheesebot
cherrypicker
cherrypickerelite/1.0
cherrypickerse/1.0
cheteam
chinaclaw
chlooe
citoid
claritybot
clark-crawler
claudebot
cliqzbot
cloud mapping
coccocbot
cocolyzebot
code87
cogentbot
cognitiveseo
cohere-ai
collector
com.plumanalytics
copernic
copier
copyrightcheck
copyscape
cosmos
cosmos
craftbot
crawl.sogou.com
crawler.feedback
crawler4j
crawling at home project
crazywebcrawler
crescent
crescent internet toolpak http ole control v.1.0
crunchbot
cshttp
curious
custo
cyotekwebcopy
databasedrivermysqli
datacha0s
dataforseo.com
dataforseobot
dblbot
demandbase-bot
demon
deusu
devil
digincore
digitalpebble
diibot
dirbuster
disco
disco
disco pump 3.0
disco pump 3.2
discobot
discofinder
discoverybot
dispatch
dittospyder
dnbcrawler-analytics
dnyzbot
domainappender
domaincrawler
domains project
domainsigmacrawler
domainsproject.org
domainstatsbot
domcopbot
dotbot
download demon
download demon/3.2.0.8
download demon/3.5.0.11
download wonder
dragonfly
drip
dsearch
dts agent
easydl
ebingbong
ecatch
ecatch/3.0
eccp/1.0
ecxi
eirgrabber
email siphon
email wolf
emailcollector
emailsiphon
emailwolf
erocrawler
evc-batch
evil
exabot
express webpictures
express webpictures (www.express-soft.com)
extlinksbot
extractor
extractorpro
extreme picture finder
eyenetie
ezooms
facebookbot
facebookscraper
fairad client
fdm
femtosearchbot
fhscan
fimap
firefox/7.0
flaming attackbot
flashget
flashget webwasher 3.2
flunky
foobot
freeuploader
frontpage
frontpage [nc,or]
fuzz
fyberspider
fyrebot
g-i-g-a-b-o-t
gaisbot
galaxybot
genieo
germcrawler
getintent
getright
getright/2.11
getright/3.1
getright/3.2
getright/3.3
getright/3.3.3
getright/3.3.4
getright/4.0.0
getright/4.1.0
getright/4.1.1
getright/4.1.2
getright/4.2
getright/4.2b (portuguxeas)
getright/4.2c
getright/4.3
getright/4.5
getright/4.5a
getright/4.5b
getright/4.5b1
getright/4.5b2
getright/4.5b3
getright/4.5b6
getright/4.5b7
getright/4.5c
getright/4.5d
getright/4.5e
getright/5.0beta1
getright/5.0beta2
getweb
getweb!
gigabot
go-ahead-got-it
go!zilla
go!zilla (www.gozilla.com)
go!zilla 3.3 (www.gozilla.com)
go!zilla 3.5 (www.gozilla.com)
gopher
gotit
gozilla
gptbot
grabber
grabnet
grafula
grapefx
grapeshotcrawler
gridbot
gt::www
haansoft
haosouspider
harvest
harvest/1.5
havij
headmasterseo
heritrix
heritrix
hloader
hloader
hmview
honolulubot
htmlparser
http::lite
httplib
httrack
httrack [nc,or]
httrack 3.0
humanlinks
humanlinks
hybridbot
iblog
id-search
idbot
idbte4m
ilsebot
image stripper
image sucker
image fetch
imagesift.com
imagesiftbot
indeedbot
indy library
indy library [nc,or]
infonavirobot
information security team infrasec scanner
infotekies
infrasec scanner
instabid
intelliseek
interget
internet ninja
internet ninja 4.0
internet ninja 5.0
internet ninja 6.0
internetmeasurement
internetseer
internetvista monitor
ips-agent
iria
irlbot
iron33/1.0.2
isitwp.com
iskanie
istellabot
iubenda-radar
jamesbot
jbrofuzz
jennybot
jetcar
jetty
jikespider
joc web spider
joomla
jorgee
justview
jyxobot
kenjin spider
kenjin.spider
keybot translation-search-machine
keyword density/0.9
keyword.density
keyword density
kinza
kozmosbot
lanshanbot
larbin
larbin
larbin (samualt9@bigfoot.com)
larbin samualt9@bigfoot.com
larbin_2.6.2 (larbin2.6.2@unspecified.mail)
larbin_2.6.2 (listonatccdotgatechdotedu)
larbin_2.6.2 (vitalbox1@hotmail.com)
larbin_2.6.2 kabura@sushi.com
larbin_2.6.2 larbin@correa.org
larbin_2.6.2 larbin2.6.2@unspecified.mail
larbin_2.6.2 listonatccdotgatechdotedu
larbin_2.6.2 vitalbox1@hotmail.com
leap
leechftp
leechget
lexibot
lftp
libweb
libweb/clshttp
libwhisker
liebaofast
lightspeedsystems
likse
linkbot
linkdexbot
linkextractorpro
linkfluence
linkpadbot
linkscan
linkscan/8.1a unix
linkscan/8.1a.unix
linksmanager
linkwalker
linqiametadatadownloaderbot
linqiarssbot
linqiascrapebot
lipperhey
lipperhey spider
litemage_walker
lmspider
lnspiderguy
ltx71
lwp-request
lwp-trivial
lwp-trivial/1.34
lwp::simple
mag-net
magnet
magpie-crawler
mail.ru_bot
majestic-seo
majestic seo
majestic12
markmonitor
markwatch
mass downloader
mass downloader/2.2
masscan
mata hari
mata.hari
mauibot
mb2345browser
meanpath bot
meanpathbot
meanpathbot
mediatoolkitbot
mediawords
megaindex.ru
metauri
mfc_tear_sample
micromessenger
microsoft url control
microsoft url control - 5.01.4511
microsoft url control - 6.00.8169
microsoft.url
microsoft data access
midown tool
miixpc
miixpc/4.2
minefield
mister pix
mister pix ii 2.01
mister pix ii 2.02a
mister pix version.dll
mister.pix
mj12bot
moblie safari
moget
moget/2.1
mojeek
mojolicious
molokaibot
morfeus fucking scanner
mozlila
mqqbrowser
mr.4x3
msfrontpage
msiecrawler
msrabot
mtrobot
muhstik-scan
musobot
name intelligence
nameprotect
navroad
nearsite
needle
nessus
net vampire
net vampire/3.0
netants
netants/1.10
netants/1.23
netants/1.24
netants/1.25
netcraft
netestate ne crawler
netlyzer
netmechanic
netspider
nettrack
netvibes
netzip
netzip downloader 1.0 win32(nov 12 1998)
netzip-downloader/1.0.62 (win32; dec 7 1998)
netzippy+(http://www.innerprise.net/usp-spider.asp)
nextgensearchbot
nibbler
nicerspro
niki-bot
nikto
nimblecrawler
nimbostratus
ninja
nmap
npbot
npbot
nuclei
nutch
obot
octopus
offline explorer
offline explorer/1.2
offline explorer/1.4
offline explorer/1.6
offline explorer/1.7
offline explorer/1.9
offline explorer/2.0
offline explorer/2.1
offline explorer/2.3
offline explorer/2.4
offline explorer/2.5
offline navigator
offline.explorer
omgili
oncrawl
openai
openai.com
openbot
openfind
openfind data gatherer
openlinkprofiler
openvas
openvas
oracle ultra search
orangebot
orangespider
outclicksbot
outfoxbot
page analyzer
page scorer
pageanalyzer
pagegrabber
pagescorer
pagething.com
pandalytics
panscient
papa foto
pavuk
pavuk
pcbrowser
pecl::http
peoplepal
perman
petalbot
phpcrawl
pi-monster
picscout
picsearch
picturefinder
piepmatz
pimonster
pixray
pleasecrawl
plumanalytics
pockey
poe-component-client-http
polaris version
probe-image-size
probethenet
propowerbot
propowerbot/2.14
prowebwalker
proximic
psbot
psbot
pu_in
pump
pxbroker
pycurl
python-urllib
queryn metasearch
queryn.metasearch
quick-crawler
radiation retriever 1.1
rainbot
rankactive
rankactivelinkbot
rankflex
rankingbot
rankingbot2
rankivabot
rankurbot
re-re
realdownload
realdownload/4.0.0.40
realdownload/4.0.0.41
realdownload/4.0.0.42
reaper
rebelmouse
recorder
redesscrapy
reget
repomonkey
repomonkey bait & tackle/v1.01
ripper
ripz
rma
rocketcrawler
rogerbot
rssingbot
s1z.ru
salesintelligent
satoristudio.net
sbider
scalaj-http
scan.lol
scanalert
scanbot
scoutjet
scrapy
screaming
screenerbot
screpybot
searchestate
searchmetricsbot
searchpreview
seekport
seekportbot
semanticjuice
semrush
semrushbot
sentibot
senutobot
seobility
seobilitybot
seocherrybot
seocompany.store
seokicks
seokicks-robot
seolyt
seolyticscrawler
seomoz
seoprofiler
seoscanners
seositecheckup
seostar
seostats
serpstatbot
sexsearcher
shodan
siphon
sistrix
site sucker
sitebeam
sitechecker.pro
sitecheckerbotcrawler
siteexplorer
siteimprove
sitelockspider
siteripz
sitesnagger
sitesucker
sitevigil
slysearch
smartdownload
smartdownload/1.2.76 (win32; apr 1 1999)
smartdownload/1.2.77 (win32; aug 17 1999)
smartdownload/1.2.77 (win32; feb 1 2000)
smartdownload/1.2.77 (win32; jun 19 2001)
smtbot
snake
snapbot
snoopy
socialrankiobot
sociscraper
sogou web spider
sogouspider
sosospider
sottopop
sp_auditbot
spacebison
spammen
spankbot
spanner
spanner
spbot
spider_bot
spider_bot/3.0
spinn3r
sputnikbot
spyfu
sqlmap
sqlworm
sqworm
sqworm/2.9.85-beta (beta_release; 20011115-775; i686-pc-linux
steeler
stripper
sucker
sucuri
superbot
superbot/3.0 (win32)
superbot/3.1 (win32)
superhttp
superhttp/1.0
surfbot
surveybot
suzuran
suzuran
swiftbot
sysscan
szukacz
szukacz/1.4
t0phackteam
t8abot
takeout
teleport
teleport pro
teleport pro/1.29
teleport pro/1.29.1590
teleport pro/1.29.1634
teleport pro/1.29.1718
teleport pro/1.29.1820
teleport pro/1.29.1847
teleportpro
telesoft
telesphoreo
telesphorep
the intraformant
the.intraformant
thenomad
thumbor
tighttwatbot
tinytestbot
titan
toata
tocrawl/urldispatcher
toweyabot
tracemyfile
trendiction
trendiction.com
trendiction.de
trendictionbot
true_robot
true_robot/1.0
turingos
turingos
turnitin
turnitinbot
turnitinbot/1.5
twengabot
twice
typhoeus
ubermetrics-technologies.com
unisterbot
upflow
url control
url_spider_pro
urly warning
urly.warning
v-bot
vacuum
vagabondo
vb project
vci
vci webviewer vci webviewer win32
velenpublicwebcrawler
vericitecrawler
vidiblescraper
virusdie
voideye
voil
voltron
voyagerx.com
wallpapers
wallpapers/3.0
wallpapershd
wasalive-bot
wbsearchbot
web image collector
web sucker
web.image.collector
web auto
web collage
web enhancer
web fetch
web fuck
web pix
web sauger
webalta
webauto
webauto/3.40 (win98; i)
webbandit
webbandit/3.50
webcapture 2.0
webcollage
webcopier
webcopier v.2.2
webcopier v2.5
webcopier v2.6
webcopier v2.7a
webcopier v2.8
webcopier v3.0
webcopier v3.0.1
webcopier v3.2
webcopier v3.2a
webdav
webemailextrac.*
webenhancer
webfetch
webfetch/2.1.0
webfuck
webgains-bot
webgo is
webimagecollector
webleacher
webmasterworldforumbot
webmeup-crawler
webpix
webpros.com
webprosbot
webreaper
webreaper [info@webreaper.net]
webreaper [webreaper@otway.com]
webreaper v9.1 - www.otway.com/webreaper
webreaper v9.7 - www.webreaper.net
webreaper v9.8 - www.webreaper.net
webreaper vwebreaper v7.3 - www,otway.com/webreaper
websauger
websauger 1.20b
websauger 1.20j
websauger 1.20k
webshag
website extractor
website extractor (http://www.asona.org)
website quester
website quester - www.asona.org
website quester - www.esalesbiz.com/extra/
website.quester
websiteextractor
websitequester
webster
webster pro
webster.pro
webstripper
webstripper/2.03
webstripper/2.10
webstripper/2.12
webstripper/2.13
webstripper/2.15
webstripper/2.16
webstripper/2.19
websucker
webwhacker
webzip
webzip
webzip/2.75 (http://www.spidersoft.com)
webzip/3.65 (http://www.spidersoft.com)
webzip/3.80 (http://www.spidersoft.com)
webzip/4.0
webzip/4.0 (http://www.spidersoft.com)
webzip/4.1 (http://www.spidersoft.com)
webzip/4.21
webzip/4.21 (http://www.spidersoft.com)
webzip/5.0
webzip/5.0 (http://www.spidersoft.com)
webzip/5.0 pr1 (http://www.spidersoft.com)
wesee
wget
wget
wget/1.5.2
wget/1.5.3
wget/1.6
wget/1.7
wget/1.8
wget/1.8.1
wget/1.8.1+cvs
wget/1.8.2
wget/1.9-beta
whack
whacker
whatweb
who.is bot
widow
winhttrack
wiseguys robot
wisenutbot
wonderbot
woobot
wotbox
wprecon
wpscan
www-collector-e
www-mechanize
www::mechanize
wwwoffle
x09mozilla
x22mozilla
xaldon webspider
xaldon webspider 2.5.b3
xaldon_webspider
xenu
xenu's
xenu's link sleuth 1.1c
xpymep1.exe
yak
youdaobot
zade
zauba
zauba.io
zermelo
zeus
zeus 11389 webster pro v2.9 win32
zeus 11652 webster pro v2.9 win32
zeus 18018 webster pro v2.9 win32
zeus 26378 webster pro v2.9 win32
zeus 30747 webster pro v2.9 win32
zeus 32297 webster pro v2.9 win32
zeus 39206 webster pro v2.9 win32
zeus 41641 webster pro v2.9 win32
zeus 44238 webster pro v2.9 win32
zeus 51070 webster pro v2.9 win32
zeus 51674 webster pro v2.9 win32
zeus 51837 webster pro v2.9 win32
zeus 63567 webster pro v2.9 win32
zeus 6694 webster pro v2.9 win32
zeus 71129 webster pro v2.9 win32
zeus 82016 webster pro v2.9 win32
zeus 82900 webster pro v2.9 win32
zeus 84842 webster pro v2.9 win32
zeus 90872 webster pro v2.9 win32
zeus 94934 webster pro v2.9 win32
zeus 95245 webster pro v2.9 win32
zeus 95351 webster pro v2.9 win32
zeus 97371 webster pro v2.9 win32
zeus link scout
zgrab
zitebot
zmeu
zoombot
zoominfobot
zumbot
zyborg

Warnings

• 5 invalid lines.