designinfo.in
robots.txt

Groups

mj12bot
rogerbot
semrushbot
exabot
dotbot
gigabot
blackwidow
chinaclaw
custo
disco
download\ demon
ecatch
eirgrabber
emailsiphon
emailwolf
express\ webpictures
extractorpro
eyenetie
flashget
getright
getweb!
go!zilla
go-ahead-got-it
grabnet
grafula
hmview
httrack
image\ stripper
image\ sucker
indy\ library
interget
internet\ ninja
jetcar
joc\ web\ spider
larbin
leechftp
mass\ downloader
midown\ tool
mister\ pix
navroad
nearsite
netants
netspider
net\ vampire
netzip
octopus
offline\ explorer
offline\ navigator
pagegrabber
papa\ foto
pavuk
pcbrowser
realdownload
reget
sitesnagger
smartdownload
superbot
superhttp
surfbot
takeout
teleport\ pro
voideye
web\ image\ collector
web\ sucker
webauto
webcopier
webfetch
webgo\ is
webleacher
webreaper
websauger
website\ extractor
website\ quester
webstripper
webwhacker
webzip
wget
widow
wwwoffle
xaldon\ webspider
zeus
wiederfreibot/1.0
cazoodlebot
dotbot/1.0
pathdefender
goodzer
megaindex.ru/2.0
ezooms
vagabondo
bspider
voltron
blexbot
linkdexbot/2.0
getintentcrawler
cliqzbot
mail.ru_bot
vegebot
adbeat_bot
jooblebot
alexibot
alvinetspider
antenne hatena
apocalxexplorerbot
asterias
backdoorbot/1.0
bizinformation
black hole
blowfish/1.0
botalot
builtbottough
bullseye/1.0
bunnyslippers
cegbfeieh
cheesebot
cherrypicker
cherrypickerelite/1.0
cherrypickerse/1.0
copyrightcheck
cosmos
crescent
crescent internet toolpak http ole control v.1.0
disco pump 3.1
dittospyder
emailcollector
erocrawler
flamingo_searchengine
foobot
harvest/1.5
hloader
httplib
httrack 3.0
humanlinks
igentia
infonavirobot
jennybot
jikespider
kenjin spider
lexibot
libweb/clshttp
linkextractorpro
linkscan/8.1a unix
linkwalker
lwp-trivial
lwp-trivial/1.34
mata hari
microsoft url control - 5.01.4511
microsoft url control - 6.00.8169
miixpc
miixpc/4.2
mister pix
mlbot
moget
moget/2.1
ms search 4.0 robot
ms search 5.0 robot
naverbot
netattache
netmechanic
nicerspro
offline explorer
openfind
openindexspider
propowerbot/2.14
prowebwalker
psbot
quepasacreep
queryn metasearch
repomonkey
rma
sightupbot
sitebot
sitesucker
sogou web spider
sosospider
spankbot
spanner
speedy
suggybot
superbot/2.6
suzuran
szukacz/1.4
teleport
telesoft
the intraformant
thenomad
tighttwatbot
titan
tocrawl/urldispatcher
toscrawler
true_robot
true_robot/1.0
turingos
turnitinbot
urlpouls
urly warning
vci
web image collector
webauto
webbandit
webbandit/3.50
webcopy
webenhancer
webmasterworldforumbot
webmirror
website extractor
website quester
webster pro
webstripper/2.02
wikiofeedbot
winhttrack
www-collector-e
xenu link sleuth/1.3.8
yacy
yrspider
zookabot
ahrefsbot
petalbot
megaindex.ru
apache-httpclient
yandexbot
yandeximages
seznambot
barkrowler
bytedance
bytespider

msnbot

No rules defined. All paths allowed.

Other Records

bingbot

No rules defined. All paths allowed.

Other Records

Warnings

• 1 invalid line.