Log In Sign Up

rednet.ag
robots.txt

Robots Exclusion Standard data for rednet.ag

Resource Scan

Scan Details

Site Domain rednet.ag
Base Domain rednet.ag
Scan Status Ok
Last Scan2024-06-09T11:16:47+00:00
Next Scan 2024-07-09T11:16:47+00:00

Last Scan

Scanned2024-06-09T11:16:47+00:00
URL https://rednet.ag/robots.txt
Redirect https://www.rednet.ag/robots.txt
Redirect Domain www.rednet.ag
Redirect Base rednet.ag
Domain IPs 78.138.119.172
Redirect IPs 78.138.119.172
Response IP 78.138.119.172
Found Yes
Hash 2cd84c2b2cd9efc7bbfd3fa1c151362e43ab43db21e207b980a7c3a9a771a178
SimHash 38b2e948ce18

Groups

abonti

Rule Path
Disallow /

acoon

Rule Path
Disallow /

acoonbot

Rule Path
Disallow /

openacoon

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

birubot

Rule Path
Disallow /

bixolabs

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

botonparade

Rule Path
Disallow /

bubing

Rule Path
Disallow /

careerbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow *

discobot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

unwindfetchor

Rule Path
Disallow /

huaweisymantecspider

Rule Path
Disallow /

iccrawler - icjobs

Rule Path
Disallow /

lcc

Rule Path
Disallow /

linguee

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

meanpathbot

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

nerdbynature.bot

Rule Path
Disallow /

obot

Rule Path
Disallow /

oneriot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

plukkie

Rule Path
Disallow /

ruby

Rule Path
Disallow /

scoutjet

Rule Path
Disallow /

search17

Rule Path
Disallow /

searchmetricsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

speedy

Rule Path
Disallow /

spinn3r

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

twenga2.com

Rule Path
Disallow /

twenga.com

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

zookabot

Rule Path
Disallow /

aboutusbot

Rule Path
Disallow /

adnormcrawler

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

aliexpress

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

baidu

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduspider-ads

Rule Path
Disallow /

baiduspider-cpro

Rule Path
Disallow /

baiduspider-favo

Rule Path
Disallow /

baiduspider-image

Rule Path
Disallow /

baiduspider-news

Rule Path
Disallow /

baiduspider-video

Rule Path
Disallow /

bizinformation

Rule Path
Disallow /

blackcatb

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

bot-pge.chlooe.com

Rule Path
Disallow /

bubing

Rule Path
Disallow /

careerbot

Rule Path
Disallow /

catchbot

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

cms crawler

Rule Path
Disallow /

coccoc

Rule Path
Disallow /

compspybot

Rule Path
Disallow /

crazywebcrawler-spider

Rule Path
Disallow /

cybeye

Rule Path
Disallow /

daumoa

Rule Path
Disallow /

discobot

Rule Path
Disallow /

domainappender

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

easouspider

Rule Path
Disallow /

euripbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

gonzo

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

haosouspider

Rule Path
Disallow /

hypercrawl

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

idmarch

Rule Path
Disallow /

iopus-web-automation

Rule Path
Disallow /

implisensebot

Rule Path
Disallow /

isowq

Rule Path
Disallow /

jamesbot

Rule Path
Disallow /

java

Rule Path
Disallow /

jobboersebot

Rule Path
Disallow /

linkdex

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /

linkpadbot

Rule Path
Disallow /

lipperhey

Rule Path
Disallow /

lipperhey spider

Rule Path
Disallow /

mail.ru

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

meanpathbot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

nerdybot

Rule Path
Disallow /

metajobbot

Rule Path
Disallow /

netestate ne crawler

Rule Path
Disallow /

netcraftsurveyagent

Rule Path
Disallow /

netseer

Rule Path
Disallow /

mfibot

Rule Path
Disallow /

mindupbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

ms search 4.0 robot

Rule Path
Disallow /

nutch

Rule Path
Disallow /

obot

Rule Path
Disallow /

onpagecheck.net

Rule Path
Disallow /

optimumform

Rule Path
Disallow /

pipl

Rule Path
Disallow /

plukkie

Rule Path
Disallow /

privacyawarebot

Rule Path
Disallow /

rankactivelinkbot

Rule Path
Disallow /

rbot

Rule Path
Disallow /

riddler

Rule Path
Disallow /

safednsbot

Rule Path
Disallow /

safetab

Rule Path
Disallow /

searchmetricsbot

Rule Path
Disallow /

seobilitybot

Rule Path
Disallow /

selfbot

Rule Path
Disallow /

semager

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

seoscanners.net

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

shopwiki

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

smtbot

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

spbot

Rule Path
Disallow /

spiderlytics

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

thumbnailagent

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

uptimebot

Rule Path
Disallow /

urlmetriken

Rule Path
Disallow /

urlpulse

Rule Path
Disallow /

urlspion

Rule Path
Disallow /

vebidoobot

Rule Path
Disallow /

vebidoobot-image

Rule Path
Disallow /

waybackarchive

Rule Path
Disallow /

webbericht.com

Rule Path
Disallow /

webinatorbot

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

websitewiki

Rule Path
Disallow /

wevikabot

Rule Path
Disallow /

woobot

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

yasni

Rule Path
Disallow /

yasnibot-image

Rule Path
Disallow /

zumbot

Rule Path
Disallow /
Back to top

Comments

  • ACHTUNG!!!
  • An dieser Datei KEINE Aenderungen vornehmen!!!
  • Dies ist eine automatisch erstellte Datei!
  • Aenderungen werden ohne Sicherung automatisch
  • ueberschrieben!
  • -------------------------------------------------
  • Version: 2019-08-07 14:48
  • Generiert: 07.08.2019 14:48
  • durch Script: esRemoteAccessSettings.class.php
  • -------------------------------------------------
  • 80legs (http://www.80legs.com/webcrawler.html)
  • Abonti (http://www.abonti.com)
  • AcoonBot (http://www.acoon.de/robot.asp)
  • Ahrefs.com (http://ahrefs.com/robot/)
  • Baiduspider (http://www.baidu.com/search/spider.htm)
  • Birubot [Bot]
  • Bixo Labs (http://bixolabs.com)
  • BLEXBot (http://webmeup.com/crawler.html)
  • BotOnParade (www.bots-on-para.de/bot.html)
  • BUbiNG (http://law.di.unimi.it/BUbiNG.html)
  • CareerBot (http://www.career-x.de/bot.html)
  • CCBot/2 (commoncrawl.org)
  • Cliqzbot (http://cliqz.com/company/cliqzbot)
  • discobot (http://discoveryengine.com/discobot.html)
  • DotBot/1.1 (http://www.opensiteexplorer.org/dotbot)
  • Exabot/3.0 (http://www.exabot.com/go/robot)
  • Gigabot (http://www.gigablast.com/spider.html)
  • Gnip [Bot] (http://www.gnip.com)
  • Huawei Symantec (http://www.huaweisymantec.com/en/IRL/spider/)
  • iCjobs Stellenangebote Jobs (www.icjobs.de/bot.htm)
  • LCC (http://corpora.informatik.uni-leipzig.de/crawler_faq.html)
  • Linguee Bot (http://www.linguee.com/bot)
  • Majestic-12 (http://www.majestic12.co.uk/bot.php)
  • meanpathbot/ (http://www.meanpath.com/meanpathbot.html)
  • MojeekBot/0.2 (http://www.mojeek.com/bot.html)
  • NerdByNature (http://www.nerdbynature.net/bot)
  • oBot/2.3.1 (http://filterdb.iss.net/crawler/)
  • OneRiot [Bot] (http://www.oneriot.com)
  • OpenLinkProfiler (http://OpenLinkProfiler.org/bot)
  • Plukkie (http://www.botje.com/plukkie.htm)
  • Ruby
  • ScoutJet (http://www.scoutjet.com/)
  • Search17Bot (http://www.search17.com/bot.php)
  • SearchmetricsBot (http://www.searchmetrics.com/en/searchmetrics-bot/)
  • SemrushBot (http://www.semrush.com/bot.html)
  • Sosospider (http://help.soso.com/webspider.htm)
  • Speedy Spider (http://www.entireweb.com/about/search_tech/speedy_spider/)
  • Spinn3r (http://spinn3r.com/robot)
  • TurnitinBot (http://www.turnitin.com/robot/crawlerinfo.html)
  • TwengaBot (http://www.twenga.com/bot.html)
  • WBSearchBot (http://www.warebay.com/bot.html)
  • XoviBot/2 (http://www.xovibot.net/)
  • Yandex (http://yandex.com/bots)
  • Zookabot (http://zookabot.com)
  • Erweiterungen des Shopbetreibers
Back to top

Warnings

  • 8 invalid lines.
Back to top