tymprod.nsfas.org.za
robots.txt

Groups

almaden
anarchie
aspseek
attach
autoemailspider
backweb
bandit
batchftp
blackwidow
buddy
bot
buddy
bumblebee
cherrypicker
chinaclaw
cicc
collector
copier
crescent
custo
da
diibot
disco
disco
pump
download\ demon
disco\ pump
download\ wonder
downloader
drip
dsurf15a
ecatch
easydl/2.99
eirgrabber
email
emailcollector
emailsiphon
emailwolf
express\ webpictures
extractorpro
eyenetie
filehound
flashget
frontpage
getright
getsmart
getweb!
gigabaz
go\!zilla
go-ahead-got-it
gotit
grabber
grabnet
grafula
grub-client
hmview
httrack
httpdown
ia_archiver
image\ stripper
image\ sucker
indy*library
indy\ library
interget
internetlinkagent
internet\ ninja
internetseer.com
iria
jbh*agent
jetcar
joc\ web\ spider
justview
larbin
leechftp
lexibot
lftp
link*sleuth
likse
link
linkwalker
mag-net
magnet
mass\ downloader
memo
microsoft.url
midown\ tool
mirror
mister\ pix
mozilla.*indy
mozilla.*newt
mozilla*msiecrawler
ms\ frontpage*
msfrontpage
msiecrawler
msproxy
navroad
nearsite
netants
netmechanic
netspider
net\ vampire
netzip
nicerspro
ninja
octopus
offline\ explorer
offline\ navigator
openfind
pagegrabber
papa\ foto
pavuk
pcbrowser
ping
pingalink
pockey
psbot
pump
qrva
realdownload
reaper
recorder
reget
scooter
seeker
siphon
sitecheck.internetseer.com
sitesnagger
slysearch
smartdownload
snake
spacebison
sproose
stripper
sucker
superbot
superhttp
surfbot
szukacz
takeout
teleport\ pro
urlspiderpro
vacuum
voideye
web\ image\ collector
web\ sucker
webauto
webcollage
webcopier
web\ downloader
webemailextrac.*
webfetch
webgo\ is
webhook
webleacher
webminer
webmirror
webreaper
websauger
website
website\ extractor
website\ quester
webster
webstripper
webwhacker
webzip
wget
whacker
widow
wwwoffle
x-tractor
xaldon\ webspider
xenu
zeus.*webster
zeus other_spam_bot

Warnings

• 2 invalid lines.