Log In Sign Up

bkdata.com
robots.txt

Robots Exclusion Standard data for bkdata.com

Resource Scan

Scan Details

Site Domain bkdata.com
Base Domain bkdata.com
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2024-08-17T16:17:30+00:00
Next Scan 2024-11-15T16:17:30+00:00

Last Successful Scan

Scanned2024-04-20T16:14:56+00:00
URL https://bkdata.com/robots.txt
Domain IPs 162.209.34.53
Response IP 162.209.34.53
Found Yes
Hash 95fb2485e7e72224d048048a9c260399727508bb46b94a3d2a4e025b102e96f4
SimHash 4c1445c0f93a

Groups

*

Rule Path
Disallow /admin/
Disallow /administrator/
Disallow /wp-admin/
Disallow /ajax/
Disallow /wp-login.php
Disallow /ajax*

bingbot

Rule Path
Disallow /research/*/*-*-*/
Disallow /research/ak/
Disallow /research/alm/
Disallow /research/aln/
Disallow /research/als/
Disallow /research/are/
Disallow /research/arw/
Disallow /research/az/
Disallow /research/cac/
Disallow /research/cae/
Disallow /research/can/
Disallow /research/cas/
Disallow /research/co/
Disallow /research/ct/
Disallow /research/dc/
Disallow /research/de/
Disallow /research/flm/
Disallow /research/fln/
Disallow /research/fls/
Disallow /research/gam/
Disallow /research/gan/
Disallow /research/gas/
Disallow /research/hi/
Disallow /research/ian/
Disallow /research/ias/
Disallow /research/id/
Disallow /research/ilc/
Disallow /research/iln/
Disallow /research/ils/
Disallow /research/inn/
Disallow /research/ins/
Disallow /research/ks/
Disallow /research/kye/
Disallow /research/kyw/
Disallow /research/lae/
Disallow /research/lam/
Disallow /research/law/
Disallow /research/ma/
Disallow /research/md/
Disallow /research/me/
Disallow /research/mie/
Disallow /research/miw/
Disallow /research/mn/
Disallow /research/moe/
Disallow /research/mow/
Disallow /research/msn/
Disallow /research/mss/
Disallow /research/mt/
Disallow /research/nce/
Disallow /research/ncm/
Disallow /research/ncw/
Disallow /research/nd/
Disallow /research/ne/
Disallow /research/nh/
Disallow /research/nj/
Disallow /research/nm/
Disallow /research/nv/
Disallow /research/nye/
Disallow /research/nyn/
Disallow /research/nys/
Disallow /research/nyw/
Disallow /research/ohn/
Disallow /research/ohs/
Disallow /research/oke/
Disallow /research/okn/
Disallow /research/okw/
Disallow /research/or/
Disallow /research/pae/
Disallow /research/pam/
Disallow /research/paw/
Disallow /research/pr/
Disallow /research/ri/
Disallow /research/sc/
Disallow /research/sd/
Disallow /research/tne/
Disallow /research/tnm/
Disallow /research/tnw/
Disallow /research/txe/
Disallow /research/txn/
Disallow /research/txs/
Disallow /research/txw/
Disallow /research/ut/
Disallow /research/vae/
Disallow /research/vaw/
Disallow /research/vt/
Disallow /research/wae/
Disallow /research/waw/
Disallow /research/wie/
Disallow /research/wiw/
Disallow /research/wvn/
Disallow /research/wvs/
Disallow /research/wy/
Disallow /research/unlock

applebot

Rule Path
Disallow /research/*/*-*-*/
Disallow /research/ak/
Disallow /research/alm/
Disallow /research/aln/
Disallow /research/als/
Disallow /research/are/
Disallow /research/arw/
Disallow /research/az/
Disallow /research/cac/
Disallow /research/cae/
Disallow /research/can/
Disallow /research/cas/
Disallow /research/co/
Disallow /research/ct/
Disallow /research/dc/
Disallow /research/de/
Disallow /research/flm/
Disallow /research/fln/
Disallow /research/fls/
Disallow /research/gam/
Disallow /research/gan/
Disallow /research/gas/
Disallow /research/hi/
Disallow /research/ian/
Disallow /research/ias/
Disallow /research/id/
Disallow /research/ilc/
Disallow /research/iln/
Disallow /research/ils/
Disallow /research/inn/
Disallow /research/ins/
Disallow /research/ks/
Disallow /research/kye/
Disallow /research/kyw/
Disallow /research/lae/
Disallow /research/lam/
Disallow /research/law/
Disallow /research/ma/
Disallow /research/md/
Disallow /research/me/
Disallow /research/mie/
Disallow /research/miw/
Disallow /research/mn/
Disallow /research/moe/
Disallow /research/mow/
Disallow /research/msn/
Disallow /research/mss/
Disallow /research/mt/
Disallow /research/nce/
Disallow /research/ncm/
Disallow /research/ncw/
Disallow /research/nd/
Disallow /research/ne/
Disallow /research/nh/
Disallow /research/nj/
Disallow /research/nm/
Disallow /research/nv/
Disallow /research/nye/
Disallow /research/nyn/
Disallow /research/nys/
Disallow /research/nyw/
Disallow /research/ohn/
Disallow /research/ohs/
Disallow /research/oke/
Disallow /research/okn/
Disallow /research/okw/
Disallow /research/or/
Disallow /research/pae/
Disallow /research/pam/
Disallow /research/paw/
Disallow /research/pr/
Disallow /research/ri/
Disallow /research/sc/
Disallow /research/sd/
Disallow /research/tne/
Disallow /research/tnm/
Disallow /research/tnw/
Disallow /research/txe/
Disallow /research/txn/
Disallow /research/txs/
Disallow /research/txw/
Disallow /research/ut/
Disallow /research/vae/
Disallow /research/vaw/
Disallow /research/vt/
Disallow /research/wae/
Disallow /research/waw/
Disallow /research/wie/
Disallow /research/wiw/
Disallow /research/wvn/
Disallow /research/wvs/
Disallow /research/wy/
Disallow /research/unlock

semrushbot

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

sistrix crawler

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

jobs.de-robot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

unisterbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

searchmetricsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

seodiver

Rule Path
Disallow /

spbot

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

meanpathbot

Rule Path
Disallow /

backlinkcrawler

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

obot

Rule Path
Disallow /

fr-crawler

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

megaindex.com

Rule Path
Disallow /

cloudservermarketspider

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

careerbot

Rule Path
Disallow /

lipperhey-kaus-australis

Rule Path
Disallow /

seoscanners.net

Rule Path
Disallow /

metajobbot

Rule Path
Disallow /

spiderbot

Rule Path
Disallow /

linkstats

Rule Path
Disallow /

jobboersebot

Rule Path
Disallow /

iccrawler

Rule Path
Disallow /

plista

Rule Path
Disallow /

domain re-animator bot

Rule Path
Disallow /

lipperhey-kaus-australis

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

coccoc

Rule Path
Disallow /

um-ic

Rule Path
Disallow /

mindupbot

Rule Path
Disallow /

sg-orbiter

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

qwantify

Rule Path
Disallow /

kraken

Rule Path
Disallow /

plukkie

Rule Path
Disallow /

safednsbot

Rule Path
Disallow /

haosouspider

Rule Path
Disallow /

openhosebot

Rule Path
Disallow /

screaming frog seo spider

Rule Path
Disallow /

thumbsniper

Rule Path
Disallow /

r6_commentreader

Rule Path
Disallow /

implisensebot

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

aihitbot

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

hubspot webcrawler

Rule Path
Disallow /

hubspot crawler

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

yandex

Rule Path
Disallow /
Back to top

Comments

  • Disallow: Sistrix
  • Disallow: Sistrix
  • Disallow: Sistrix
  • Disallow: SEOkicks-Robot
  • Disallow: jobs.de-Robot
  • Backlink Analysis
  • Bot der Leipziger Unister Holding GmbH
  • http://moz.com/products
  • http://www.searchmetrics.com
  • http://www.majestic12.co.uk/projects/dsearch/mj12bot.php
  • http://www.domaintools.com/webmasters/surveybot.php
  • http://www.seodiver.com/bot
  • http://openlinkprofiler.org/bot
  • http://www.wotbox.com/bot/
  • http://www.opensiteexplorer.org/dotbot
  • http://moz.com/researchtools/ose/dotbot
  • http://www.meanpath.com/meanpathbot.html
  • http://www.backlinktest.com/crawler.html
  • http://www.brandwatch.com/magpie-crawler/
  • http://filterdb.iss.net/crawler/
  • http://webmeup-crawler.com
  • https://megaindex.com/crawler
  • http://www.cloudservermarket.com
  • http://www.trendiction.de/de/publisher/bot
  • http://www.exalead.com
  • http://www.career-x.de/bot.html
  • https://www.lipperhey.com/en/about/
  • https://www.lipperhey.com/en/about/
  • https://turnitin.com/robot/crawlerinfo.html
  • http://help.coccoc.com/
  • ubermetrics-technologies.com
  • datenbutler.de
  • http://searchgears.de/uber-uns/crawling-faq.html
  • http://commoncrawl.org/faq/
  • https://www.qwant.com/
  • http://linkfluence.net/
  • http://www.botje.com/plukkie.htm
  • https://www.safedns.com/searchbot
  • http://www.haosou.com/help/help_3_2.html
  • http://www.haosou.com/help/help_3_2.html
  • http://www.moz.com/dp/rogerbot
  • User-agent: rogerbot
  • Disallow: /
  • http://www.openhose.org/bot.html
  • http://www.screamingfrog.co.uk/seo-spider/
  • http://thumbsniper.com
  • http://www.radian6.com/crawler
  • http://cliqz.com/company/cliqzbot
  • https://www.aihitdata.com/about
  • http://www.trendiction.com/en/publisher/bot
  • http://warebay.com/bot.html
  • hubspot
  • hubspot
Back to top

Warnings

  • 2 invalid lines.
Back to top