nba-live.com robots.txt

aitcsrobot/1.1

Rule	Path
Disallow	/

Rule

Path

Disallow

/

alexibot

Rule	Path
Disallow	/

Rule

Path

Disallow

/

aqua_products

Rule	Path
Disallow	/

Rule

Path

Disallow

/

arachnophilia

Rule	Path
Disallow	/

Rule

Path

Disallow

/

architextspider

Rule	Path
Disallow	/

Rule

Path

Disallow

/

askjeeves

Rule	Path
Disallow	/

Rule

Path

Disallow

/

aspider/0.09

Rule	Path
Disallow	/

Rule

Path

Disallow

/

asterias

Rule	Path
Disallow	/

Rule

Path

Disallow

/

auresys/1.0

Rule	Path
Disallow	/

Rule

Path

Disallow

/

b2w/0.1

Rule	Path
Disallow	/

Rule

Path

Disallow

/

backdoorbot/1.0

Rule	Path
Disallow	/

Rule

Path

Disallow

/

backrub/.

Rule	Path
Disallow	/

Rule

Path

Disallow

/

becomebot

Rule	Path
Disallow	/

Rule

Path

Disallow

/

big brother

Rule	Path
Disallow	/

Rule

Path

Disallow

/

bizbot003

Rule	Path
Disallow	/

Rule

Path

Disallow

/

bizbot04 kirk.overleaf.com

Rule	Path
Disallow	/

Rule

Path

Disallow

/

blackwidow

Rule	Path
Disallow	/

Rule

Path

Disallow

/

blowfish/1.0

Rule	Path
Disallow	/

Rule

Path

Disallow

/

bookmark search tool

Rule	Path
Disallow	/

Rule

Path

Disallow

/

botalot

Rule	Path
Disallow	/

Rule

Path

Disallow

/

bspider/1.0 libwww-perl/0.40

Rule	Path
Disallow	/

Rule

Path

Disallow

/

builtbottough

Rule	Path
Disallow	/

Rule

Path

Disallow

/

bullseye/1.0

Rule	Path
Disallow	/

Rule

Path

Disallow

/

bunnyslippers

Rule	Path
Disallow	/

Rule

Path

Disallow

/

cactvs chemistry spider

Rule	Path
Disallow	/

Rule

Path

Disallow

/

californiabrownspider

Rule	Path
Disallow	/

Rule

Path

Disallow

/

checkbot/x.xx lwp/5.x

Rule	Path
Disallow	/

Rule

Path

Disallow

/

cheesebot

Rule	Path
Disallow	/

Rule

Path

Disallow

/

cherrypicker

Rule	Path
Disallow	/

Rule

Path

Disallow

/

cherrypickerelite/1.0

Rule	Path
Disallow	/

Rule

Path

Disallow

/

cherrypickerse/1.0

Rule	Path
Disallow	/

Rule

Path

Disallow

/

cmc/0.01

Rule	Path
Disallow	/

Rule

Path

Disallow

/

combine/0.0

Rule	Path
Disallow	/

Rule

Path

Disallow

/

computingsite robi/1.0 (robi@computingsite.com)

Rule	Path
Disallow	/

Rule

Path

Disallow

/

conceptbot/0.3

Rule	Path
Disallow	/

Rule

Path

Disallow

/

copernic

Rule	Path
Disallow	/

Rule

Path

Disallow

/

copyrightcheck

Rule	Path
Disallow	/

Rule

Path

Disallow

/

cosmos

Rule	Path
Disallow	/

Rule

Path

Disallow

/

crescent

Rule	Path
Disallow	/

Rule

Path

Disallow

/

crescent internet toolpak http ole control v.1.0

Rule	Path
Disallow	/

Rule

Path

Disallow

/

crescent internet toolpak http ole control v.1.0

Rule	Path
Disallow	/

Rule

Path

Disallow

/

cs-hkust-indexserver/1.0

Rule	Path
Disallow	/

Rule

Path

Disallow

/

cyberspyder (amant@www.cyberspyder.com)

Rule	Path
Disallow	/

Rule

Path

Disallow

/

cyberspyder/2.1

Rule	Path
Disallow	/

Rule

Path

Disallow

/

deweb/1.01

Rule	Path
Disallow	/

Rule

Path

Disallow

/

digimarc cgireader/1.0

Rule	Path
Disallow	/

Rule

Path

Disallow

/

digimarc webreader/1.2

Rule	Path
Disallow	/

Rule

Path

Disallow

/

dittospyder

Rule	Path
Disallow	/

Rule

Path

Disallow

/

dlw3robot/x.y (in tclx by http://hplyot.obspm.fr/~dl/)

Rule

Path

Disallow

/

dragonbot/1.0 libwww/5.0

Rule

Path

Disallow

/

due to a deficiency in java it's not currently possible to set the user-agent.

Rule

Path

Disallow

/

dumbot

Rule

Path

Disallow

/

duppies

Rule

Path

Disallow

/

ei*net/0.1 libwww/0.1

Rule

Path

Disallow

/

eit-link-verifier-robot/0.2

Rule

Path

Disallow

/

emacs-w3/v[0-9\.]+

Rule

Path

Disallow

/

emailcollector

Rule

Path

Disallow

/

emailsiphon

Rule

Path

Disallow

/

emailsiphon

Rule

Path

Disallow

/

emailwolf

Rule

Path

Disallow

/

emc spider

Rule

Path

Disallow

/

enigmabot

Rule

Path

Disallow

/

enterprise_search

Rule

Path

Disallow

/

enterprise_search/1.0

Rule

Path

Disallow

/

erocrawler

Rule

Path

Disallow

/

es

Rule

Path

Disallow

/

esirover v1.0

Rule

Path

Disallow

/

esismartspider/2.0

Rule

Path

Disallow

/

explorer

Rule

Path

Disallow

/

explorersearch

Rule

Path

Disallow

/

extractorpro

Rule

Path

Disallow

/

extractorpro

Rule

Path

Disallow

/

fairad client

Rule

Path

Disallow

/

felixide/1.0

Rule

Path

Disallow

/

fido/0.9 harvest/1.4.pl2

Rule

Path

Disallow

/

fish-search-robot

Rule

Path

Disallow

/

flaming attackbot

Rule

Path

Disallow

/

foobot

Rule

Path

Disallow

/

freecrawl

Rule

Path

Disallow

/

funnelweb-1.0

Rule

Path

Disallow

/

gaisbot

Rule

Path

Disallow

/

gcreep/1.0

Rule

Path

Disallow

/

gestalticonoclast/1.0 libwww-fm/2.17

Rule

Path

Disallow

/

getright/4.2

Rule

Path

Disallow

/

geturl.rexx v1.05

Rule

Path

Disallow

/

golem/1.1

Rule

Path

Disallow

/

gromit/1.0

Rule

Path

Disallow

/

grub

Rule

Path

Disallow

/

grub-client

Rule

Path

Disallow

/

gulliver/1.1

Rule

Path

Disallow

/

hämähäkki/0.2

Rule

Path

Disallow

/

happybot (gserver.kw.net)

Rule

Path

Disallow

/

harvest/1.5

Rule

Path

Disallow

/

hatena antenna

Rule

Path

Disallow

/

heraspider

Rule

Path

Disallow

/

hloader

Rule

Path

Disallow

/

htdig/3.0b3

Rule

Path

Disallow

/

htmlgobble v2.2

Rule

Path

Disallow

/

httplib

Rule

Path

Disallow

/

humanlinks

Rule

Path

Disallow

/

ia_archiver

Rule

Path

Disallow

/

ia_archiver/1.6

Rule

Path

Disallow

/

ibot/1.0 libwww-perl/0.40

Rule

Path

Disallow

/

incywincy/1.0b1

Rule

Path

Disallow

/

infonavirobot

Rule

Path

Disallow

/

informant

Rule

Path

Disallow

/

infoseek robot 1.0

Rule

Path

Disallow

/

infoseek sidewinder

Rule

Path

Disallow

/

infospiders/0.1

Rule

Path

Disallow

/

ingrid/0.1

Rule

Path

Disallow

/

inspectorwww/1.0 http://www.greenpac.com/inspectorwww.html

Rule

Path

Disallow

/

iron33/1.0.2

Rule

Path

Disallow

/

israelisearch/1.0

Rule

Path

Disallow

/

iti spider

Rule

Path

Disallow

/

jcrawler/0.2

Rule

Path

Disallow

/

jeeves

Rule

Path

Disallow

/

jennybot

Rule

Path

Disallow

/

jetbot

Rule

Path

Disallow

/

jetbot/1.0

Rule

Path

Disallow

/

jobot/0.1alpha libwww-perl/4.0

Rule

Path

Disallow

/

jubiirobot

Rule

Path

Disallow

/

jumpstation

Rule

Path

Disallow

/

katipo/1.0

Rule

Path

Disallow

/

kdd-explorer/0.1

Rule

Path

Disallow

/

kenjin spider

Rule

Path

Disallow

/

keyword density/0.9

Rule

Path

Disallow

/

kit-fireball/2.0 libwww/5.0a

Rule

Path

Disallow

/

ko_yappo_robot/1.0.4(http://yappo.com/info/robot.html)

Rule

Path

Disallow

/

labelgrab/1.1

Rule

Path

Disallow

/

larbin

Rule

Path

Disallow

/

lexibot

Rule

Path

Disallow

/

libertech-rover www.libertech.com?

Rule

Path

Disallow

/

libweb/clshttp

Rule

Path

Disallow

/

linkextractorpro

Rule

Path

Disallow

/

linklooker

Rule

Path

Disallow

/

linkscan/8.1a unix

Rule

Path

Disallow

/

linkwalker

Rule

Path

Disallow

/

linkwalker

Rule

Path

Disallow

/

lnspiderguy

Rule

Path

Disallow

/

logo.gif crawler

Rule

Path

Disallow

/

looksmart

Rule

Path

Disallow

/

lwp-trivial

Rule

Path

Disallow

/

lwp-trivial/1.34

Rule

Path

Disallow

/

lycos/x.x

Rule

Path

Disallow

/

lycos_spider_(t-rex)

Rule

Path

Disallow

/

magpie/1.0

Rule

Path

Disallow

/

mata hari

Rule

Path

Disallow

/

mediafox/x.y

Rule

Path

Disallow

/

merritt/1.0

Rule

Path

Disallow

/

merzscope

Rule

Path

Disallow

/

metagopher

Rule

Path

Disallow

/

microsoft url control

Rule

Path

Disallow

/

microsoft url control - 5.01.4511

Rule

Path

Disallow

/

microsoft url control - 6.00.8169

Rule

Path

Disallow

/

miixpc

Rule

Path

Disallow

/

miixpc/4.2

Rule

Path

Disallow

/

mister pix

Rule

Path

Disallow

/

moget

Rule

Path

Disallow

/

moget/2.1

Rule

Path

Disallow

/

momspider/1.00 libwww-perl/0.40

Rule

Path

Disallow

/

monster/vx.x.x -$type ($ostype)

Rule

Path

Disallow

/

motor/0.2

Rule

Path

Disallow

/

mozilla

Rule

Path

Disallow

/

mozilla

Rule

Path

Disallow

/

mozilla/2.0 (compatible fouineur v2.0;fouineur.9bit.qc.ca)

Rule

Path

Disallow

/

mozilla/3

Rule

Path

Disallow

/

mozilla/3.0 (black widow v1.1.0; linux 2.0.27; dec31 1997 12:25:00

Rule

Path

Disallow

/

mozilla/3.0 (compatible;miner;mailto:miner@miner.com.br)

Rule

Path

Disallow

/

mozilla/4

Rule

Path

Disallow

/

mozilla/4.0 (compatible; bullseye; windows 95)

Rule

Path

Disallow

/

mozilla/4.0 (compatible; msie 4.0; windows 2000)

Rule

Path

Disallow

/

mozilla/4.0 (compatible; msie 4.0; windows 95)

Rule

Path

Disallow

/

mozilla/4.0 (compatible; msie 4.0; windows 98)

Rule

Path

Disallow

/

mozilla/4.0 (compatible; msie 4.0; windows nt)

Rule

Path

Disallow

/

mozilla/4.0 (compatible; msie 4.0; windows xp)

Rule

Path

Disallow

/

mozilla/4.0 (compatible; msie 5.0; win32)

Rule

Path

Disallow

/

mozilla/4.0 (compatible; msie 5.01; windows nt 5.0)

Rule

Path

Disallow

/

mozilla/4.0 (compatible; msie 5.5; windows nt 5.0)

Rule

Path

Disallow

/

mozilla/4.0 (compatible; msie 6.0; windows nt 5.0)

Rule

Path

Disallow

/

mozilla/4.5 (compatible; msie 4.01; windows 95)

Rule

Path

Disallow

/

mozilla/4.5 (compatible; msie 4.01; windows 98)

Rule

Path

Disallow

/

mozilla/4.5 (compatible; msie 4.01; windows nt)

Rule

Path

Disallow

/

mozilla/4.61 [en] (win98; i)

Rule

Path

Disallow

/

mozilla/4.78 [en] (windows nt 5.0; u)

Rule

Path

Disallow

/

mozilla/5

Rule

Path

Disallow

/

msiecrawler

Rule

Path

Disallow

/

muscatferret

Rule

Path

Disallow

/

mwdsearch/0.1

Rule

Path

Disallow

/

mycnnspider

Rule

Path

Disallow

/

naver

Rule

Path

Disallow

/

nec-meshexplorer

Rule

Path

Disallow

/

nederland.zoek

Rule

Path

Disallow

/

netants

Rule

Path

Disallow

/

netcarta cyberpilot pro

Rule

Path

Disallow

/

netmechanic

Rule

Path

Disallow

/

netmechanic

Rule

Path

Disallow

/

netscoop/1.0 libwww/5.0a

Rule

Path

Disallow

/

nhsewalker/3.0

Rule

Path

Disallow

/

nicerspro

Rule

Path

Disallow

/

no

Rule

Path

Disallow

/

no

Rule

Path

Disallow

/

no

Rule

Path

Disallow

/

no

Rule

Path

Disallow

/

nomad-v2.x

Rule

Path

Disallow

/

none

Rule

Path

Disallow

/

none

Rule

Path

Disallow

/

northstar

Rule

Path

Disallow

/

not available

Rule

Path

Disallow

/

nutch

Rule

Path

Disallow

/

occam/1.0

Rule

Path

Disallow

/

offline explorer

Rule

Path

Disallow

/

ogspider

Rule

Path

Disallow

/

open text site crawler v1.0

Rule

Path

Disallow

/

openbot

Rule

Path

Disallow

/

openfind

Rule

Path

Disallow

/

openfind data gathere

Rule

Path

Disallow

/

oracle ultra search

Rule

Path

Disallow

/

orbsearch/1.0

Rule

Path

Disallow

/

packrat/1.0

Rule

Path

Disallow

/

patric/0.01a

Rule

Path

Disallow

/

peregrinator-mathematics/0.7

Rule

Path

Disallow

/

perman

Rule

Path

Disallow

/

pgp-ka/1.2

Rule

Path

Disallow

/

pioneer

Rule

Path

Disallow

/

processor/0.0alpha libwww-perl/0.20

Rule

Path

Disallow

/

propowerbot/2.14

Rule

Path

Disallow

/

prowebwalker

Rule

Path

Disallow

/

psbot

Rule

Path

Disallow

/

python-urllib

Rule

Path

Disallow

/

queryn metasearch

Rule

Path

Disallow

/

radiation retriever 1.1

Rule

Path

Disallow

/

repomonkey

Rule

Path

Disallow

/

repomonkey bait & tackle/v1.01

Rule

Path

Disallow

/

resume robot

Rule

Path

Disallow

/

rma

Rule

Path

Disallow

/

road runner: imagescape robot (lim@cs.leidenuniv.nl)

Rule

Path

Disallow

/

robbie/0.1

Rule

Path

Disallow

/

robot du crim 1.0a

Rule

Path

Disallow

/

root/0.1

Rule

Path

Disallow

/

roverbot

Rule

Path

Disallow

/

sandbot

Rule

Path

Disallow

/

scooter

Rule

Path

Disallow

/

scooter/1.0

Rule

Path

Disallow

/

searchpreview

Rule

Path

Disallow

/

senrigan/xxxxxx

Rule

Path

Disallow

/

sg-scout

Rule

Path

Disallow

/

shai'hulud

Rule

Path

Disallow

/

simbot/1.0

Rule

Path

Disallow

/

sitesnagger

Rule

Path

Disallow

/

sitetech-rover

Rule

Path

Disallow

/

slowbot

Rule

Path

Disallow

/

slurp/2.0

Rule

Path

Disallow

/

snooper/b97_01

Rule

Path

Disallow

/

solbot/1.0 lwp/5.07

Rule

Path

Disallow

/

sootle

Rule

Path

Disallow

/

spankbot

Rule

Path

Disallow

/

spanner

Rule

Path

Disallow

/

spanner/1.0 (linux 2.0.27 i586)

Rule

Path

Disallow

/

spyder3.microsys.com

Rule

Path

Disallow

/

stanford

Rule

Path

Disallow

/

stanford comp sci

Rule

Path

Disallow

/

stanford compclub

Rule

Path

Disallow

/

stanford compsciclub

Rule

Path

Disallow

/

stanford spiderboys

Rule

Path

Disallow

/

statfetcher/1.0

Rule

Path

Disallow

/

summycrawler

Rule

Path

Disallow

/

surfbot

Rule

Path

Disallow

/

suzuran

Rule

Path

Disallow

/

szukacz/1.4

Rule

Path

Disallow

/

tarantula/1.0

Rule

Path

Disallow

/

tarspider

Rule

Path

Disallow

/

teachersoft/1.0 libwww/2.17

Rule

Path

Disallow

/

teleport

Rule

Path

Disallow

/

teleportpro

Rule

Path

Disallow

/

telesoft

Rule

Path

Disallow

/

templeton/

Rule

Path

Disallow

/

teoma

Rule

Path

Disallow

/

the intraformant

Rule

Path

Disallow

/

thenomad

Rule

Path

Disallow

/

titan/0.1

Rule

Path

Disallow

/

titin/0.2

Rule

Path

Disallow

/

tocrawl/urldispatcher

Rule

Path

Disallow

/

true_robot

Rule

Path

Disallow

/

true_robot/1.0

Rule

Path

Disallow

/

turingos

Rule

Path

Disallow

/

ucsd-crawler

Rule

Path

Disallow

/

url control

Rule

Path

Disallow

/

url_spider_pro

Rule

Path

Disallow

/

urlck/1.2.3

Rule

Path

Disallow

/

urly warning

Rule

Path

Disallow

/

valkyrie/1.0 libwww-perl/0.40

Rule

Path

Disallow

/

vci

Rule

Path

Disallow

/

vci webviewer vci webviewer win32

Rule

Path

Disallow

/

victoria/1.0

Rule

Path

Disallow

/

vision-search/3.0'

Rule

Path

Disallow

/

vwbot_k/4.2

Rule

Path

Disallow

/

w3index

Rule

Path

Disallow

/

w3index

Rule

Path

Disallow

/

w3m2/x.xxx

Rule

Path

Disallow

/

w3mir

Rule

Path

Disallow

/

web image collector

Rule

Path

Disallow

/

webauto

Rule

Path

Disallow

/

webbandit

Rule

Path

Disallow

/

webbandit/3.50

Rule

Path

Disallow

/

webcopier

Rule

Path

Disallow

/

webcopy/

Rule

Path

Disallow

/

webcrawler/3.0 robot libwww/5.0a

Rule

Path

Disallow

/

webenhancer

Rule

Path

Disallow

/

webferret

Rule

Path

Disallow

/

weblayers/0.0

Rule

Path

Disallow

/

weblinker/0.0 libwww-perl/0.1

Rule

Path

Disallow

/

webmasterworld extractor

Rule

Path

Disallow

/

webmasterworldforumbot

Rule

Path

Disallow

/

webmoose/0.0.0000

Rule

Path

Disallow

/

webs@recruit.co.jp

Rule

Path

Disallow

/

websauger

Rule

Path

Disallow

/

website quester

Rule

Path

Disallow

/

webster pro

Rule

Path

Disallow

/

webstripper

Rule

Path

Disallow

/

webvac

Rule

Path

Disallow

/

webvac/1.0

Rule

Path

Disallow

/

webwalk

Rule

Path

Disallow

/

webwalker

Rule

Path

Disallow

/

webwalker/1.10

Rule

Path

Disallow

/

webwatch

Rule

Path

Disallow

/

webzip

Rule

Path

Disallow

/

webzip/4.0

Rule

Path

Disallow

/

wget

Rule

Path

Disallow

/

wget/1.4.0

Rule

Path

Disallow

/

wget/1.5.3

Rule

Path

Disallow

/

wget/1.6

Rule

Path

Disallow

/

whowhere robot

Rule

Path

Disallow

/

wired-digital-newsbot/1.5

Rule

Path

Disallow

/

wobot/1.0 from 206.214.202.45

Rule

Path

Disallow

/

www collector

Rule

Path

Disallow

/

www.freeloader.com.

Rule

Path

Disallow

/

wwwc/0.25 (win95)

Rule

Path

Disallow

/

www-collector-e

Rule

Path

Disallow

/

wwwwanderer v3.0

Rule

Path

Disallow

/

xenu's

Rule

Path

Disallow

/

xenu's link sleuth 1.1c

Rule

Path

Disallow

/

xget/0.7

Rule

Path

Disallow

/

yes

Rule

Path

Disallow

/

zeus

Rule

Path

Disallow

/

zeus 32297 webster pro v2.9 win32

Rule

Path

Disallow

/

zeus link scout

Rule

Path

Disallow

/

botrighthere

Rule

Path

Disallow

/

webzip

Rule

Path

Disallow

/

larbin

Rule

Path

Disallow

/

b2w/0.1

Rule

Path

Disallow

/

copernic

Rule

Path

Disallow

/

psbot

Rule

Path

Disallow

/

python-urllib

Rule

Path

Disallow

/

netmechanic

Rule

Path

Disallow

/

url_spider_pro

Rule

Path

Disallow

/

cherrypicker

Rule

Path

Disallow

/

emailcollector

Rule

Path

Disallow

/

emailsiphon

Rule

Path

Disallow

/

webbandit

Rule

Path

Disallow

/

emailwolf

Rule

Path

Disallow

/

extractorpro

Rule

Path

Disallow

/

copyrightcheck

Rule

Path

Disallow

/

crescent

Rule

Path

Disallow

/

sitesnagger

Rule

Path

Disallow

/

prowebwalker

Rule

Path

Disallow

/

cheesebot

Rule

Path

Disallow

/

lnspiderguy

Rule

Path

Disallow

/

alexibot

Rule

Path

Disallow

/

teleport

Rule

Path

Disallow

/

teleportpro

Rule

Path

Disallow

/

miixpc

Rule

Path

Disallow

/

telesoft

Rule

Path

Disallow

/

website quester

Rule

Path

Disallow

/

webzip

Rule

Path

Disallow

/

moget/2.1

Rule

Path

Disallow

/

webzip/4.0

Rule

Path

Disallow

/

webstripper

Rule

Path

Disallow

/

websauger

Rule

Path

Disallow

/

webcopier

Rule

Path

Disallow

/

netants

Rule

Path

Disallow

/

mister pix

Rule

Path

Disallow

/

webauto

Rule

Path

Disallow

/

thenomad

Rule

Path

Disallow

/

www-collector-e

Rule

Path

Disallow

/

rma

Rule

Path

Disallow

/

libweb/clshttp

Rule

Path

Disallow

/

asterias

Rule

Path

Disallow

/

httplib

Rule

Path

Disallow

/

turingos

Rule

Path

Disallow

/

spanner

Rule

Path

Disallow

/

infonavirobot

Rule

Path

Disallow

/

harvest/1.5

Rule

Path

Disallow

/

bullseye/1.0

Rule

Path

Disallow

/

mozilla/4.0 (compatible; bullseye; windows 95)

Rule

Path

Disallow

/

crescent internet toolpak http ole control v.1.0

Rule

Path

Disallow

/

cherrypickerse/1.0

Rule

Path

Disallow

/

cherrypickerelite/1.0

Rule

Path

Disallow

/

webbandit/3.50

Rule

Path

Disallow

/

nicerspro

Rule

Path

Disallow

/

microsoft url control - 5.01.4511

Rule

Path

Disallow

/

dittospyder

Rule

Path

Disallow

/

foobot

Rule

Path

Disallow

/

spankbot

Rule

Path

Disallow

/

botalot

Rule

Path

Disallow

/

lwp-trivial/1.34

Rule

Path

Disallow

/

lwp-trivial

Rule

Path

Disallow

/

bunnyslippers

Rule

Path

Disallow

/

microsoft url control - 6.00.8169

Rule

Path

Disallow

/

urly warning

Rule

Path

Disallow

/

wget/1.6

Rule

Path

Disallow

/

wget/1.5.3

Rule

Path

Disallow

/

wget

Rule

Path

Disallow

/

linkwalker

Rule

Path

Disallow

/

cosmos

Rule

Path

Disallow

/

moget

Rule

Path

Disallow

/

hloader

Rule

Path

Disallow

/

humanlinks

Rule

Path

Disallow

/

linkextractorpro

Rule

Path

Disallow

/

offline explorer

Rule

Path

Disallow

/

mata hari

Rule

Path

Disallow

/

lexibot

Rule

Path

Disallow

/

web image collector

Rule

Path

Disallow

/

the intraformant

Rule

Path

Disallow

/

true_robot/1.0

Rule

Path

Disallow

/

true_robot

Rule

Path

Disallow

/

blowfish/1.0

Rule

Path

Disallow

/

jennybot

Rule

Path

Disallow

/

miixpc/4.2

Rule

Path

Disallow

/

builtbottough

Rule

Path

Disallow

/

propowerbot/2.14

Rule

Path

Disallow

/

backdoorbot/1.0

Rule

Path

Disallow

/

tocrawl/urldispatcher

Rule

Path

Disallow

/

suzuran

Rule

Path

Disallow

/

tighttwatbot

Rule

Path

Disallow

/

vci webviewer vci webviewer win32

Rule

Path

Disallow

/

vci

Rule

Path

Disallow

/

szukacz/1.4

Rule

Path

Disallow

/

openfind data gatherer

Rule

Path

Disallow

/

openfind

Rule

Path

Disallow

/

xenu's link sleuth 1.1c

Rule

Path

Disallow

/

xenu's

Rule

Path

Disallow

/

zeus

Rule

Path

Disallow

/

repomonkey bait & tackle/v1.01

Rule

Path

Disallow

/

repomonkey

Rule

Path

Disallow

/

microsoft url control

Rule

Path

Disallow

/

openbot

Rule

Path

Disallow

/

url control

Rule

Path

Disallow

/

zeus link scout

Rule

Path

Disallow

/

zeus 32297 webster pro v2.9 win32

Rule

Path

Disallow

/

webster pro

Rule

Path

Disallow

/

erocrawler

Rule

Path

Disallow

/

linkscan/8.1a unix

Rule

Path

Disallow

/

keyword density/0.9

Rule

Path

Disallow

/

kenjin spider

Rule

Path

Disallow

/

iron33/1.0.2

Rule

Path

Disallow

/

bookmark search tool

Rule

Path

Disallow

/

getright/4.2

Rule

Path

Disallow

/

fairad client

Rule

Path

Disallow

/

gaisbot

Rule

Path

Disallow

/

aqua_products

Rule

Path

Disallow

/

radiation retriever 1.1

Rule

Path

Disallow

/

flaming attackbot

Rule

Path

Disallow

/

Site Domain	nba-live.com
Base Domain	nba-live.com
Scan Status	Ok
Last Scan	2024-06-26T18:13:17+00:00
Next Scan	2024-07-03T18:13:17+00:00

Scanned	2024-06-26T18:13:17+00:00
URL	https://nba-live.com/robots.txt
Domain IPs	67.231.250.222
Response IP	67.231.250.222
Found	Yes
Hash	39e972ee3812e40229e3e11831701e34429e199b8d6d4eb974227358a9cc9923
SimHash	c1bd529aebd3

nba-live.comrobots.txt

Resource Scan

Scan Details

Last Scan

Groups

aitcsrobot/1.1

alexibot

aqua_products

arachnophilia

architextspider

askjeeves

aspider/0.09

asterias

auresys/1.0

b2w/0.1

backdoorbot/1.0

backrub/*.*

becomebot

big brother

bizbot003

bizbot04 kirk.overleaf.com

blackwidow

blowfish/1.0

bookmark search tool

botalot

bspider/1.0 libwww-perl/0.40

builtbottough

bullseye/1.0

bunnyslippers

cactvs chemistry spider

californiabrownspider

checkbot/x.xx lwp/5.x

cheesebot

cherrypicker

cherrypickerelite/1.0

cherrypickerse/1.0

cmc/0.01

combine/0.0

computingsite robi/1.0 (robi@computingsite.com)

conceptbot/0.3

copernic

copyrightcheck

cosmos

crescent

crescent internet toolpak http ole control v.1.0

crescent internet toolpak http ole control v.1.0

cs-hkust-indexserver/1.0

cyberspyder (amant@www.cyberspyder.com)

cyberspyder/2.1

deweb/1.01

digimarc cgireader/1.0

digimarc webreader/1.2

dittospyder

dlw3robot/x.y (in tclx by http://hplyot.obspm.fr/~dl/)

dragonbot/1.0 libwww/5.0

due to a deficiency in java it's not currently possible to set the user-agent.

dumbot

duppies

ei*net/0.1 libwww/0.1

eit-link-verifier-robot/0.2

emacs-w3/v[0-9\.]+

emailcollector

emailsiphon

emailsiphon

emailwolf

emc spider

enigmabot

enterprise_search

enterprise_search/1.0

erocrawler

es

esirover v1.0

esismartspider/2.0

explorer

explorersearch

extractorpro

extractorpro

fairad client

felixide/1.0

fido/0.9 harvest/1.4.pl2

nba-live.com
robots.txt

backrub/.