Log In Sign Up

computerdirect.co.nz
robots.txt

Robots Exclusion Standard data for computerdirect.co.nz

Resource Scan

Scan Details

Site Domain computerdirect.co.nz
Base Domain computerdirect.co.nz
Scan Status Ok
Last Scan2025-11-23T09:21:43+00:00
Next Scan 2025-12-23T09:21:43+00:00

Last Scan

Scanned2025-11-23T09:21:43+00:00
URL https://computerdirect.co.nz/robots.txt
Domain IPs 121.99.241.244
Response IP 121.99.241.244
Found Yes
Hash 15cde8515a3fbcfa4aae1f2356a5bd300af3185c5c41425a0eeff24cf43e97a5
SimHash 79519591cb77

Groups

cfm-searchbot(http://www.cfm-search.com)

Rule Path
Disallow /

cjnetworkquality

Rule Path
Disallow /

googlebot-image

Rule Path
Disallow

hatena antenna

Rule Path
Disallow /

infosniff sniffer

Rule Path
Disallow /

msnbot

Rule Path
Disallow

npbot

Rule Path
Disallow /

nutch

Rule Path
Disallow /

nutchcvs

Rule Path
Disallow /

nutchorg

Rule Path
Disallow /

semanticdiscovery

Rule Path
Disallow /

tutorgig

Rule Path
Disallow /

tutorial crawler

Rule Path
Disallow /

ubicrawler

Rule Path
Disallow /

webfilter robot 1.0

Rule Path
Disallow /

webmasterworldforumbot

Rule Path
Disallow /

*

Rule Path
Disallow
Back to top

Comments

  • Provided courtesy of www.GaryKeith.com
  • Created on July 22, 2003 at 8:19:48 PM GMT
  • The bots in this section are asked not to spider any of this site.
  • Persistent unauthorized bots may be banned by host and/or IP.
  • everyone else is asked to avoid the following folders and files:
Back to top