Log In Sign Up

diskografie.cz
robots.txt

Robots Exclusion Standard data for diskografie.cz

Resource Scan

Scan Details

Site Domain diskografie.cz
Base Domain diskografie.cz
Scan Status Ok
Last Scan2024-06-04T22:55:22+00:00
Next Scan 2024-06-11T22:55:22+00:00

Last Scan

Scanned2024-06-04T22:55:22+00:00
URL https://diskografie.cz/robots.txt
Domain IPs 91.121.89.64
Response IP 91.121.89.64
Found Yes
Hash 16de7ad8a05dee37b4d01b55d371226c29ac931e7d2e682064fbd543775d9029
SimHash 5217c43ac529

Groups

blexbot

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduspider+

Rule Path
Disallow /

gingercrawler

Rule Path
Disallow /

nutch

Rule Path
Disallow /

urlbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

scoutjet

Rule Path
Disallow /

yandex

Rule Path
Disallow /

speedy spider

Rule Path
Disallow /

accelobot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

discobot

Rule Path
Disallow /

aboutusbot

Rule Path
Disallow /

vocus

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

plukkie

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

avant browser (http://www.avantbrowser.com)

Rule Path
Disallow /

http://ahrefs.com/robot

Rule Path
Disallow /

http://www.genieo.com/webfilter.html

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

linkdex.com

Rule Path
Disallow /

linkdex.com/v2.0

Rule Path
Disallow /

semrushbot/0.95

Rule Path
Disallow /

mj12bot/v1.4.3

Rule Path
Disallow /

java/1.6.0_04

Rule Path
Disallow /

js-kit url resolver, http://js-kit.com/

Rule Path
Disallow /

js-kit

Rule Path
Disallow /

netshelter contentscan

Rule Path
Disallow /

netshelter

Rule Path
Disallow /

presto/2.8.119

Rule Path
Disallow /

presto

Rule Path
Disallow /

searchmetricsbot

Rule Path
Disallow /

python-requests/1.2.0

Rule Path
Disallow /

cpython/2.7.3

Rule Path
Disallow /

mozilla/5.0 moreover/5.1 (+http://www.moreover.com; webmaster@moreover.com)

Rule Path
Disallow /

mozilla/4.0 (compatible; win32; winhttp.winhttprequest.5)

Rule Path
Disallow /

mozilla/5.0 (compatible; sosospider/2.0; +http://help.soso.com/webspider.htm)

Rule Path
Disallow /

mozilla/5.0 (x11; compatible; semantic-visions.com crawler; httpclient 3.1)

Rule Path
Disallow /

mozilla/5.0 (compatible; yyspider; +http://www.yunyun.com/spider.html)

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

mozilla/3.0 (compatible; indy library)

Rule Path
Disallow /

pagepeeker2

Rule Path
Disallow /

mozilla/4.0 (windows 98; us) opera 10.00 [en]

Rule Path
Disallow /

mozilla/5.0 (compatible; linux x86_64; mail.ru_bot/2.0; +http://go.mail.ru/help/robots)

Rule Path
Disallow /

siteexplorer

Rule Path
Disallow /

ruby

Rule Path
Disallow /

abonti

Rule Path
Disallow /

mozilla/5.0 (compatible; seokicks-robot +http://www.seokicks.de/robot.html)

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

boardreader

Rule Path
Disallow /

fairshare-http://fairshare.cc

Rule Path
Disallow /

verticalleap-pagestatusbot/2.0(+http://www.apolloinsights.com/)

Rule Path
Disallow /

linkedin/7.1 cfnetwork/672.0.8 darwin/14.0.0

Rule Path
Disallow /

linkedin/7.1

Rule Path
Disallow /

pcore-http/v0.25.0

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

dataforseobot/1.0

Rule Path
Disallow /

barkrowler/0.9

Rule Path
Disallow /
Back to top

Comments

  • bad-bots
Back to top

Warnings

  • 14 invalid lines.
Back to top