myadsclassified.com
robots.txt

Groups

*

mediapartners-google

googlebot

semrushbot
semrushbot
semrushbot-sa
deusu
netcraftsurveyagent
mj12botahrefsbot
spider
yandex
yandex.com
yandexbot/3.0
megaindex.ru/2.0
buzzbot/1.0 (buzzbot; http://www.buzzstream.com; buzzbot@buzzstream.com)
megaindex.ru
mozilla/4.0 (compatible; msie 6.0; windows nt 5.1; sv1)
mail.ru_bot
mozilla/5.0 (compatible; linux x86_64; mail.ru_bot/2.0; +http://go.mail.ru/help/robots)
mozilla/5.0 (compatible; findxbot/1.0; +http://www.findxbot.com)
mozilla/5.0 (compatible; xovibot/2.0; +http://www.xovibot.net/)
xovibot
blexbot
semantic-visions.com crawler
mozilla/5.0 (x11; compatible; semantic-visions.com crawler; httpclient 3.1)
mozilla/4.0 (compatible; win32; winhttp.winhttprequest.5)
mozilla/5.0 (compatible; sosospider/2.0; +http://help.soso.com/webspider.htm)
seokicks
seokicks-robot
boardreader
mozilla/5.0 (compatible; megaindex.ru/2.0; +http://megaindex.com/crawler)
baiduspider
baiduspider+
baiduspider+(+http://www.baidu.com/search/spider.htm)
baiduspider/2.0;+http://www.baidu.com/search/spider.html
baiduspider/2.0
mozilla/5.0(compatible; baiduspider/2.0; +http://www.baidu.com/search/spider.html)
wesee:ads/picturebot (http://www.wesee.com/bot/)
ahrefsbot
dotbot
mail.ru
ravencrawler
psbot/0.1
twiceler www.cuill.com/robots.html
twiceler-0.9 http://www.cuill.com/twiceler/robot.html
dwaarbot+(dwaarbot@dwaar.com)
cazoodlebot
giga?mega.bot/1.0; +http://www.giga?mega.net/bot.html
mj12bot
baidu
speedy
iiitbot/1.1 (indian language web search engine; http://webkhoj.iiit.net; pvvpr at iiit dot ac dot in)
boitho.com-dc
ccbot
seekbot
pete-spider light
ltx71 - (http://ltx71.com/)
ltx71
appi
ahrefsbot
sosospider
boeserseospider
stackrambler
sphider
jyxobot/1
jyxobot
gigabot
dotbot
dotbot/1.0
shopwiki
naverbot-1.0
naverbot
grub-client
grub
webzip
larbin
b2w/0.1
copernic
psbot
python-urllib
netmechanic
url_spider_pro
cherrypicker
emailcollector
emailsiphon
webbandit
emailwolf
extractorpro
copyrightcheck
crescent
sitesnagger
prowebwalker
cheesebot
lnspiderguy
mozilla/4.0 (compatible; msie 4.0; windows nt)
mozilla/4.0 (compatible; msie 4.0; windows 95)
mozilla/4.0 (compatible; msie 4.0; windows 98)
mozilla/4.0 (compatible; msie 4.0; windows xp)
mozilla/4.0 (compatible; msie 4.0; windows 2000)
alexibot
teleport
teleportpro
miixpc
telesoft
website quester
moget/2.1
webzip/4.0
webstripper
websauger
webcopier
netants
mister pix
thenomad
www-collector-e
rma
libweb/clshttp
asterias
httplib
turingos
spanner
infonavirobot
harvest/1.5
bullseye/1.0
mozilla/4.0 (compatible; bullseye; windows 95)
webbandit/3.50
nicerspro
dittospyder
webmasterworldforumbot
spankbot
botalot
lwp-trivial/1.34
lwp-trivial
bunnyslippers
urly warning
wget/1.6
wget/1.5.3
wget
linkwalker
cosmos
moget
hloader
humanlinks
linkextractorpro
offline explorer
mata hari
lexibot
web image collector
the intraformant
true_robot/1.0
true_robot
blowfish/1.0
jennybot
miixpc/4.2
builtbottough
propowerbot/2.14
backdoorbot/1.0
tocrawl/urldispatcher
webenhancer
suzuran
vci webviewer vci webviewer win32
vci
szukacz/1.4
queryn metasearch
openfind data gathere
openfind
xenu's link sleuth 1.1c
xenu's link sleuth 1.2d
xenu's
zeus
repomonkey bait & tackle/v1.01
repomonkey
openbot
url control
zeus link scout
zeus 32297 webster pro v2.9 win32
webster pro
erocrawler
linkscan/8.1a unix
keyword density/0.9
kenjin spider
iron33/1.0.2
bookmark search tool
getright/4.2
fairad client
gaisbot
aqua_products
radiation retriever 1.1
webmasterworld extractor
flaming attackbot
oracle ultra search
perman
sootle
turnitinbot
haste
ezooms
ezooms/1.0
mozilla/5.0 (compatible; ezooms/1.0; ezooms.bot[at]gmail[dot]com)
omniexplorer_bot
chatalogica.com cgi script

Other Records

Warnings

• 4 invalid lines.