Log In Sign Up

leadersummaries.com
robots.txt

Robots Exclusion Standard data for leadersummaries.com

Resource Scan

Scan Details

Site Domain leadersummaries.com
Base Domain leadersummaries.com
Scan Status Ok
Last Scan2024-05-21T04:33:08+00:00
Next Scan 2024-06-20T04:33:08+00:00

Last Scan

Scanned2024-05-21T04:33:08+00:00
URL https://leadersummaries.com/robots.txt
Redirect https://www.leadersummaries.com/robots.txt
Redirect Domain www.leadersummaries.com
Redirect Base leadersummaries.com
Domain IPs 34.78.242.115
Redirect IPs 34.78.242.115
Response IP 34.78.242.115
Found Yes
Hash 38243c8f06c1bf773293cfa4150ce767ecad2780ae3c6e544d90099bcf328950
SimHash 327e354668fa

Groups

*

Rule Path
Allow *
Disallow */backend/
Disallow *ReturnUrl%3D
Disallow */_partials/
Disallow *corporation_nv%3D

ucrawlr

Rule Path
Disallow /

siteexplorer

Rule Path
Disallow /

netestate ne crawler

Rule Path
Disallow /

metauri

Rule Path
Disallow /

crawler4j

Rule Path
Disallow /

megaindex.ru/2.0

Rule Path
Disallow /

megaindex.ru/

Rule Path
Disallow /

mozilla/5.0+(compatible;+megaindex.ru/2.0;++http://megaindex.com/crawler)

Rule Path
Disallow /

mozilla/5.0+(compatible;+megaindex.ru/2.0;++http://megaindex.com/crawler)

Rule Path
Disallow /

megaindex.ru/2.0

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

ltx71+-+(http://ltx71.com/)

Rule Path
Disallow /

http://ltx71.com/

Rule Path
Disallow /

nrlcorpusbuilder/nutch-1.9

Rule Path
Disallow /

nutch-1.9

Rule Path
Disallow /

nutch-1.4/nutch-1.4

Rule Path
Disallow /

nutch-1.4

Rule Path
Disallow /

nutch-1.10

Rule Path
Disallow /

gigablastopensource

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

yoozbot

Rule Path
Disallow /

haosouspider

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

mail.ru

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

sogou blog

Rule Path
Disallow /

sogou inst spider

Rule Path
Disallow /

sogou news spider

Rule Path
Disallow /

sogou orion spider

Rule Path
Disallow /

sogou spider2

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

istellabot

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduspider-image

Rule Path
Disallow /

rambler

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

sputnikbot

Rule Path
Disallow /

plukkie

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

deusu

Rule Path
Disallow /

coccoc

Rule Path
Disallow /

coccocbot-image

Rule Path
Disallow /

coccocbot-web

Rule Path
Disallow /

daumoa

Rule Path
Disallow /

dazoobot

Rule Path
Disallow /

ichiro

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

sistrix crawler

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

searchmetricsbot

Rule Path
Disallow /

spiderling

Rule Path
Disallow /

checkmarknetwork/1.0 (+http://www.checkmarknetwork.com/spider.html)

Rule Path
Disallow /

findxbot

Rule Path
Disallow /

yacybot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

domainstatsbot/1.0 (http://domainstats.io/our-bot)

Rule Path
Disallow /

panscient.com

Rule Path
Disallow /

genieo

Rule Path
Disallow /

mozilla/5.0 (compatible; genieo/1.0 http://www.genieo.com/webfilter.html)

Rule Path
Disallow /

vebidoobot

Rule Path
Disallow /

bubing

Rule Path
Disallow /

uptimebot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

safednsbot

Rule Path
Disallow /

dataprovider

Rule Path
Disallow /

ziba

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

rankingbot2

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.leadersummaries.com/sitemap
Back to top

Comments

  • Allow
  • yooz.ir iranian search engine
  • so.com chinese search engine 360
  • seznam.cz czech search engine
  • mail.ru russian search engine
  • mojeek.com search engine
  • sogou.com chinese search engine
  • istella.it italian search engine
  • cliqz.com german in-product search engine
  • baidu.com chinese search engine
  • rambler.ru russian search engine
  • soso.com chinese search engine
  • sputnik.ru russian search engine
  • botje.nl dutch search engine
  • youdao.com chinese search engine
  • naver.com south korean search engine
  • Distributed Search Engine http://www.majestic12.co.uk/projects/dsearch/mj12bot.php
  • deusu.de german search engine
  • coccoc.com vietnamese search engine
  • daum.net korean search engine
  • dazoo.fr french search engine
  • goo.ne.jp japanese search engine
  • Sistrix seo tool
  • http://moz.com/products
  • Ahrefs Backlink Analysis
  • http://webmeup-crawler.com
  • https://www.semrush.com/bot/
  • http://www.searchmetrics.com/searchmetricsbot/
  • text corpora for various languages https://nlp.fi.muni.cz/projects/biwec/
  • Trademark monitoring
  • Danish startup beta search engine
  • yacy.net p2p search software
  • SeoProfiler http://openlinkprofiler.org/bot
  • DomainStatsBot
  • Panscient crawls the web and turns it into structured databases http://panscient.com/faq.htm
  • Genieo Web filter Information for web masters http://www.genieo.com/webfilter.html
  • German web crawler https://blog.vebidoo.de/vebidoobot/
  • crawler under development http://law.di.unimi.it/BUbiNG.html
  • Server Monitoring https://uptime.com/uptimebot
  • Search engine http://www.exalead.com/search/webmasterguide
  • Search engine http://www.trendiction.com/en/publisher/bot
  • Search engine https://www.safedns.com/es/searchbot
  • https://www.dataprovider.com/spider/
  • research project https://zibalinks.com/
  • Search engine http://www.wotbox.com/bot/
  • Search engine http://www.wotbox.com/bot/
Back to top

Warnings

  • 10 invalid lines.
Back to top