Log In Sign Up

iacsp.com
robots.txt

Robots Exclusion Standard data for iacsp.com

Resource Scan

Scan Details

Site Domain iacsp.com
Base Domain iacsp.com
Scan Status Ok
Last Scan2024-09-01T13:02:32+00:00
Next Scan 2024-10-01T13:02:32+00:00

Last Scan

Scanned2024-09-01T13:02:32+00:00
URL https://iacsp.com/robots.txt
Domain IPs 192.252.146.19
Response IP 192.252.146.19
Found Yes
Hash 9304d58b334c2ecdf75dd02d708af306eedbcbf3145eb6006654156cec13c941
SimHash a65f52120393

Groups

googlebot

Rule Path
Disallow /cgi-bin
Disallow /SANDTRAP
Disallow /poll/
Disallow /images/

*

Rule Path
Disallow /cgi-bin/

mozilla/3.01 (hotwired-test/0.1)

Rule Path
Disallow /cgi-bin/

slurp

Rule Path
Disallow /cgi-bin/

ultraseek

Rule Path
Disallow /cgi-bin/

smallbear

Rule Path
Disallow /cgi-bin/

webferret

Rule Path
Disallow

arachnophilia

Rule Path
Disallow

architextspider

Rule Path
Disallow

aspider/0.09

Rule Path
Disallow

auresys/1.0

Rule Path
Disallow

backrub/*.*

Rule Path
Disallow

big brother

Rule Path
Disallow

blackwidow

Rule Path
Disallow

bspider/1.0 libwww-perl/0.40

Rule Path
Disallow

cactvs chemistry spider

Rule Path
Disallow

digimarc cgireader/1.0

Rule Path
Disallow

checkbot/x.xx lwp/5.x

Rule Path
Disallow

cmc/0.01

Rule Path
Disallow

combine/0.0

Rule Path
Disallow

conceptbot/0.3

Rule Path
Disallow

crescent internet toolpak http ole control v.1.0

Rule Path
Disallow

root/0.1

Rule Path
Disallow

cs-hkust-indexserver/1.0

Rule Path
Disallow

cyberspyder/2.1

Rule Path
Disallow

deweb/1.01

Rule Path
Disallow

dragonbot/1.0 libwww/5.0

Rule Path
Disallow

eit-link-verifier-robot/0.2

Rule Path
Disallow

emacs-w3/v[0-9\.]+

Rule Path
Disallow

emailsiphon

Rule Path
Disallow

emc spider

Rule Path
Disallow

explorersearch

Rule Path
Disallow

explorer

Rule Path
Disallow

extractorpro

Rule Path
Disallow

felixide/1.0

Rule Path
Disallow

hazel's ferret web hopper,

Rule Path
Disallow

esirover v1.0

Rule Path
Disallow

fido/0.9 harvest/1.4.pl2

Rule Path
Disallow

hämähäkki/0.2

Rule Path
Disallow

kit-fireball/2.0 libwww/5.0a

Rule Path
Disallow

fish-search-robot

Rule Path
Disallow

mozilla/2.0 (compatible fouineur v2.0; fouineur.9bit.qc.ca)

Rule Path
Disallow

robot du crim 1.0a

Rule Path
Disallow

freecrawl

Rule Path
Disallow

funnelweb-1.0

Rule Path
Disallow

gcreep/1.0

Rule Path
Disallow

geturl.rexx v1.05

Rule Path
Disallow

golem/1.1

Rule Path
Disallow

gromit/1.0

Rule Path
Disallow

gulliver/1.1

Rule Path
Disallow

yes

Rule Path
Disallow

aitcsrobot/1.1

Rule Path
Disallow

wired-digital-newsbot/1.5

Rule Path
Disallow

htdig/3.0b3

Rule Path
Disallow

htmlgobble v2.2

Rule Path
Disallow

no

Rule Path
Disallow

ibm_planetwide,

Rule Path
Disallow

gestalticonoclast/1.0 libwww-fm/2.17

Rule Path
Disallow

ingrid/0.1

Rule Path
Disallow

incywincy/1.0b1

Rule Path
Disallow

informant

Rule Path
Disallow

infoseek robot 1.0

Rule Path
Disallow

infoseek sidewinder

Rule Path
Disallow

infospiders/0.1

Rule Path
Disallow

websiteoutlook

Rule Path
Disallow /
Back to top

Comments

  • robots, goaway
  • this bans robots from our cgi-bin and othe places they should not be
Back to top

Warnings

  • 4 invalid lines.
Back to top