Log In Sign Up

pigdog.org
robots.txt

Robots Exclusion Standard data for pigdog.org

Resource Scan

Scan Details

Site Domain pigdog.org
Base Domain pigdog.org
Scan Status Ok
Last Scan2025-06-05T21:50:20+00:00
Next Scan 2025-07-05T21:50:20+00:00

Last Scan

Scanned2025-06-05T21:50:20+00:00
URL https://pigdog.org/robots.txt
Redirect https://www.pigdog.org/robots.txt
Redirect Domain www.pigdog.org
Redirect Base pigdog.org
Domain IPs 173.236.245.59
Redirect IPs 173.236.245.59
Response IP 173.236.245.59
Found Yes
Hash 201076a0fef3a564ab6cca11bbfe664a0614493477051471a633ef3507688d88
SimHash 255c8241dc97

Groups

webclipping.com

Rule Path
Disallow /

arianna.iol.it linux/2.2.17-14smp (linux)

Rule Path
Disallow /

slysearch

Rule Path
Disallow /

larbin_2.6.2 larbin2.6.2@unspecified.mail

Rule Path
Disallow /

larbin_2.6.2

Rule Path
Disallow /

larbin_2.6.1 larbin2.6.2@unspecified.mail

Rule Path
Disallow /

larbin_2.6.1

Rule Path
Disallow /

libwww-perl/5.5.3

Rule Path
Disallow /

libwww-perl/5.53

Rule Path
Disallow /

libwww-perl

Rule Path
Disallow /

python-urllib/1.10

Rule Path
Disallow /

python-urllib

Rule Path
Disallow /

java1.3.0

Rule Path
Disallow /

java1.4.0

Rule Path
Disallow /

java

Rule Path
Disallow /

winampmpeg/2.00 larbin@unspecified.mail

Rule Path
Disallow /

winampmpeg

Rule Path
Disallow /

msie-5.13 larbin@unspecified.mail

Rule Path
Disallow /

opera/6.01 larbin2.6.2@unspecified.mail

Rule Path
Disallow /

opera/6.01 larbin@unspecified.mail

Rule Path
Disallow /

mozilla/5.0 larbin2.6.2@unspecified.mail

Rule Path
Disallow /

bumblebee@relevare.com

Rule Path
Disallow /

zeus 64087 webster pro v2.9 win32

Rule Path
Disallow /

netresearchserver/2.3(loopimprovements.com/robot.html)

Rule Path
Disallow /

netresearchserver

Rule Path
Disallow /

openfind data gatherer, openbot/3.0+(robot-response@openfind.com.tw;+http://www.openfind.com.tw/robot.html)

Rule Path
Disallow /

openbot

Rule Path
Disallow /

http://www.almaden.ibm.com/cs/crawler

Rule Path
Disallow /

netmechanic v3.0

Rule Path
Disallow /

netmechanic

Rule Path
Disallow /

ariadne rpt-httpclient/0.3-3

Rule Path
Disallow /

ariadne

Rule Path
Disallow /

mozilla/4.0 compatible zyborg/1.0 (zyborg@wisenutbot.com; http://www.wisenutbot.com)

Rule Path
Disallow /

zyborg

Rule Path
Disallow /

appie 1.1 (www.walhello.com)

Rule Path
Disallow /

appie

Rule Path
Disallow /

search.ch v1.4.2 (spiderman@search.ch; http://www.search.ch)

Rule Path
Disallow /

search.ch

Rule Path
Disallow /

pingalink monitoring services 1.0 (http://www.pingalink.com)

Rule Path
Disallow /

pingalink monitoring services

Rule Path
Disallow /

pingalink

Rule Path
Disallow /

aweb-ii v3.4se/amigaos-3.9

Rule Path
Disallow /

aweb-ii

Rule Path
Disallow /

*

Rule Path
Disallow /stats/
Disallow /cgi_bin/
Disallow /private/
Back to top

Comments

  • Exclude some bots.
  • http://www.plagiarism.org/crawler/robotinfo.html
  • Fuck off, snitchbot!
  • People who just set up dork-ass library bots they downloaded
  • off the Innurnet, and don't even bother to ID themselves,
  • are ASS. Go fuck yourself.
  • Oooh, aren't you the clever H4X0R, now. Fuck off and die, also.
  • Welcome to the Innernet. You might want to read the HTTP spec before
  • fucking with our Web site. Your user-agent ID is bad, which suggests
  • that you don't know shit, and you're a sloppy programmer. Go away.
  • this is not email, d00d.
  • whitespace, d00d.
  • ID first, d00d. Then a space, then a comment.
  • I don't care if you _ARE_ IBM. Jeebus. Stupid scientists.
  • Gosh, so close.
  • No spaces. First the ID, then a space, then a comment.
  • You suck.
  • slash, not a space
  • bad version string
  • No spaces.
  • fuck amigas, anyways
  • OK, everybody else: don't go here.
Back to top