kilokalorie.cz
robots.txt

Robots Exclusion Standard data for kilokalorie.cz

Resource Scan

Scan Details

Site Domain kilokalorie.cz
Base Domain kilokalorie.cz
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-10-07T07:53:05+00:00
Next Scan 2024-10-14T07:53:05+00:00

Last Successful Scan

Scanned2024-09-29T07:52:48+00:00
URL https://kilokalorie.cz/robots.txt
Domain IPs 89.221.217.219
Response IP 89.221.217.219
Found Yes
Hash df916c70c099b742776f19829103382d09f08d0caa59704038d46921e5f7efe5
SimHash 6adbe45266c1

Groups

*

Rule Path
Disallow /oou
Disallow /cookies

seznambot
sklikbot

Rule Path
Disallow /oou
Disallow /cookies

bingbot

Rule Path
Disallow /oou
Disallow /cookies

Other Records

Field Value
crawl-delay 300

ahrefsbot
amazonbot
archive.org_bot
baiduspider
barkrowler
bleriot
blexbot
bubing
ccbot
cliqzbot
dataforseobot
dataprovider
desu
deusu
dnyzbot
dotbot
embedly
exabot
genieo
heritrix
ia_archiver
lcc
linguee
linkdexbot
linkpadbot
ltx71
mauibot
megaindex.ru
mj12bot
msnbot
netseer
qwantify
riddler
seekport
semrushbot
semrushbot-sa
seokicks
seokicks-robot
serpstatbot
smtbot
sogou
spbot
spiderling
surdotlybot
turnitinbot
uptimebot
velenpublicwebcrawler
xovibot
yandexbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://kilokalorie.cz/sitemap.xml

Warnings

  • 1 invalid line.
  • `request-rate` is not a known field.
  • `robot-version` is not a known field.