dgt.es
robots.txt

Groups

*
googlebot
googlebot-news
googlebot-image
googlebot-video
addthis.com disallow: /
admantx disallow: /
ahrefsbot disallow: /
alexibot disallow: /
aqua_products disallow: /
bdcbot disallow: /
bender disallow: /
bixocrawler disallow: /
bl.uk_lddc_bot disallow: /
blexbot disallow: /
bubing disallow: /
cliqzbot disallow: /
cncdialer disallow: /
crawler4j disallow: /
crystalsemanticsbot disallow: /
cyberalert disallow: /
digext disallow: /
discobot disallow: /
discoverybot disallow: /
dloader disallow: /
dloader(naverrobot) disallow: /
doc disallow: /
dotbot disallow: /
download ninja disallow: /
dts agent disallow: /
exabot disallow: /
ezooms disallow: /
fairshare disallow: /
fetch disallow: /
flamingo_searchengine disallow: /
genieo disallow: /
gigabot disallow: /
grub-client disallow: /
heritrix disallow: /
heritrix/3.3.0 disallow: /
httrack disallow: /
ia_archiver disallow: /
integromedb disallow: /
istellabot disallow: /
jikespider disallow: /
jyxobot disallow: /
k2spider disallow: /
kimengi disallow: /
kimengi/nineconnections.com disallow: /
larbin disallow: /
lexxebot/1.0 disallow: /
libwww disallow: /
linko disallow: /
livelapbot disallow: /
magpie-crawler disallow: /
maxthon disallow: /
metauri disallow: /
microsoft.url.control disallow: /
mj12bot disallow: /
moreover disallow: /
moreoverbot disallow: /
msiecrawler disallow: /
nabot disallow: /
naverbot disallow: /
nerdbynature.bot disallow: /
netestate ne crawler disallow: /
netseer crawler disallow: /
newscan disallow: /
nextgensearchbot disallow: /
npbot disallow: /
nutch disallow: /
offline explorer disallow: /
omgilibot disallow: /
orthogaffe disallow: /
piplbot disallow: /
pixray-seeker disallow: /
proximic disallow: /
psbot disallow: /
queryseekerspider disallow: /
rogerbot disallow: /
seokicks disallow: /
seokicks-robot disallow: /
sitebot disallow: /
sitebot/0.1 disallow: /
sitecheck.internetseer.com disallow: /
sitesnagger disallow: /
slurp disallow: /
sogou disallow: /
sosospider disallow: /
spbot disallow: /
spinn3r disallow: /
teleport disallow: /
teleportpro disallow: /
trendictionbot disallow: /
trovitbot disallow: /
turnitinbot disallow: /
ubicrawler disallow: /
umbot-ln disallow: /
unisterbot disallow: /
universalfeedparser disallow: /
wbsearchbot disallow: /
webcopier disallow: /
webreaper disallow: /
webstripper disallow: /
webzip disallow: /
wesee:search disallow: /
wget disallow: /
wotbot disallow: /
wotbox disallow: /
xenu disallow: /
xenu's link sleuth 1.1c disallow: /
xenu's disallow: /
yasni disallow: /
yahoo! slurp china disallow: /
zao disallow: /
zealbot disallow: /
zyborg disallow: /
zeus disallow: /
zeus link scout disallow: /
zeus 32297 webster pro v2.9 win32 disallow: /
b2w/0.1 disallow: /
backdoorbot/1.0 disallow: /
bookmark search tool disallow: /
botalot disallow: /
botrighthere disallow: /
builtbottough disallow: /
bullseye/1.0 disallow: /
bunnyslippers disallow: /
cherrypicker disallow: /
cherrypickerse/1.0 disallow: /
cherrypickerelite/1.0 disallow: /
cheesebot disallow: /
copernic disallow: /
cosmos disallow: /
crescent disallow: /
crescent internet toolpak http ole control v.1.0 disallow: /
dittospyder disallow: /
emailcollector disallow: /
emailsiphon disallow: /
emailwolf disallow: /
erocrawler disallow: /
extractorpro disallow: /
fairad client disallow: /
fast disallow: /
flaming attackbot disallow: /
foobot disallow: /
gaisbot disallow: /
getright/4.2 disallow: /
harvest/1.5 disallow: /
hloader disallow: /
httplib disallow: /
httrack 3.0 disallow: /
humanlinks disallow: /
infonavirobot disallow: /
iron33/1.0.2 disallow: /
jennybot disallow: /
jyxobot/1 disallow: /
kenjin spider disallow: /
keyword density/0.9 disallow: /
lexibot disallow: /
libweb/clshttp disallow: /
linkextractorpro disallow: /
linkscan/8.1a unix disallow: /
linkwalker disallow: /
lnspiderguy disallow: /
lwp-trivial/1.34 disallow: /
lwp-trivial disallow: /
mata hari disallow: /
miixpc disallow: /
miixpc/4.2 disallow: /
mister pix disallow: /
moget disallow: /
moget/2.1 disallow: /
netants disallow: /
nicerspro disallow: /
openbot disallow: /
openfind data gatherer disallow: /
openfind disallow: /
oracle ultra search disallow: /
perman disallow: /
prowebwalker disallow: /
propowerbot/2.14 disallow: /
python-urllib disallow: /
queryn metasearch disallow: /
radiation retriever 1.1 disallow: /
repomonkey bait & tackle/v1.01 disallow: /
repomonkey disallow: /
rma disallow: /
searchpreview disallow: /
spankbot disallow: /
spanner disallow: /
suzuran disallow: /
szukacz/1.4 disallow: /
telesoft disallow: /
the intraformant disallow: /
tighttwatbot disallow: /
thenomad disallow: /
tocrawl/urldispatcher disallow: /
true_robot/1.0 disallow: /
true_robot disallow: /
turingos disallow: /
turnitinbot/1.5 disallow: /
url control disallow: /
url_spider_pro disallow: /
urly warning disallow: /
vci webviewer vci webviewer win32 disallow: /
vci disallow: /
webbandit disallow: /
webbandit/3.50 disallow: /
webcapture 2.0 disallow: /
webcopier v3.2a disallow: /
webcopier v.2.2 disallow: /
web image collector disallow: /
webenhancer disallow: /
websauger disallow: /
website quester disallow: /
webster pro disallow: /
webzip/4.0 disallow: /
webzip/4.21 disallow: /
webzip/5.0 disallow: /
wget/1.6 disallow: /
wget/1.5.3 disallow: /
wget disallow: /
www-collector-e disallow: /
archive.org_bot
msnbot

No rules defined. All paths allowed.

Other Records

slurp

No rules defined. All paths allowed.

Other Records

Other Records

Comments

• ___
• _____ /\__\
• /::\ \ /:/ _/_ ___
• /:/\:\ \ /:/ /\ \ /\__\
• /:/ \:\__\ /:/ /::\ \ /:/ /
• /:/__/ \:|__| /:/__\/\:\__\ /:/__/
• \:\ \ /:/ / \:\ \ /:/ / /::\ \
• \:\ /:/ / \:\ /:/ / /:/\:\ \
• \:\/:/ / \:\/:/ / \/__\:\ \
• \::/ / \::/ / \:\__\
• \/__/ \/__/ \/__/
• ____ _ _ _ _
• | _ \ ___ | |__ ___ | |_ ___ | |___ _| |_
• | |_) / _ \| '_ \ / _ \| __/ __|| __\ \/ / __|
• | _ < (_) | |_) | (_) | |_\__ \| |_ > <| |_
• |_| \_\___/|_.__/ \___/ \__|___(_)__/_/\_\\__|
• Evitar rastreo de Bad Bots
• conf especifica bots

Warnings

• 1 invalid line.
• `visit-time` is not a known field.