Log In Sign Up

transresult.net
robots.txt

Robots Exclusion Standard data for transresult.net

Resource Scan

Scan Details

Site Domain transresult.net
Base Domain transresult.net
Scan Status Ok
Last Scan2024-09-20T10:45:41+00:00
Next Scan 2024-10-20T10:45:41+00:00

Last Scan

Scanned2024-09-20T10:45:41+00:00
URL http://transresult.net/robots.txt
Domain IPs 188.64.57.13
Response IP 188.64.57.13
Found Yes
Hash 992bb841a0bf298d53cdf5c3ff98abbf0b18cb88cb0d12d530afd0c908800a6c
SimHash 6e574f18c487

Groups

psbot

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

scoutjet

Rule Path
Disallow /

wget

Rule Path
Disallow /

eurobot

Rule Path
Disallow /

twiceler

Rule Path
Disallow /

gaisbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

www-mechanize

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

gonzo

Rule Path
Disallow /

gonzo*

Rule Path
Disallow /

sapphirewebcrawler

Rule Path
Disallow /

cabot

Rule Path
Disallow /

acontbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

catchbot

Rule Path
Disallow /

webrankspider

Rule Path
Disallow /

yacy

Rule Path
Disallow /

yacybot

Rule Path
Disallow /

mail.ru

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

surveybot_ignoreip

Rule Path
Disallow /

yanga worldsearch bot

Rule Path
Disallow /

oozbot

Rule Path
Disallow /

plukkie

Rule Path
Disallow /

http://www.uni-koblenz.de/~flocke/robot-info.txt

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

iis3bot

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

citenikbot

Rule Path
Disallow /

charlotte

Rule Path
Disallow /

exabot

Rule Path
Disallow /

vedensbot

Rule Path
Disallow /

lexxebot

Rule Path
Disallow /

violabot

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

tagoobot

Rule Path
Disallow /

cityreview

Rule Path
Disallow /

euripbot

Rule Path
Disallow /

butterfly

Rule Path
Disallow /

isara-search

Rule Path
Disallow /

jyxobot

Rule Path
Disallow /

mlbot

Rule Path
Disallow /

libwww-perl

Rule Path
Disallow /

nutch

Rule Path
Disallow /

nutch-agent

Rule Path
Disallow /

panscient.com

Rule Path
Disallow /

botonparade

Rule Path
Disallow /

yandex

Rule Path
Disallow /

jobs.de-robot

Rule Path
Disallow /

*

Rule Path
Disallow /netz/
Disallow /controls/
Disallow /App_Themes/
Disallow /WebResource.axd
Disallow /Login.aspx
Back to top

Comments

  • robots.txt
  • Created: Mon, 10 Oct 2009 13:02:03 GMT
  • Please note: There are a lot of pages on this site, and there are
  • some misbehaved spiders out there. If you're
  • irresponsible, your access to the site may be blocked.
  • http://eurobot.ayell.eu/
  • http://www.cuill.com/twiceler/robot.html
  • aka Nutch
  • http://www.amfibi.com/cabot/
  • http://spider.acont.de/
  • http://www.setooz.com/oozbot.html
  • http://www.botje.com/plukkie.htm
  • http://www.mojeek.com/bot.html
  • specialists -------------------
Back to top

Warnings

  • 2 invalid lines.
Back to top