vylustime.cz
robots.txt

Robots Exclusion Standard data for vylustime.cz

Resource Scan

Scan Details

Site Domain vylustime.cz
Base Domain vylustime.cz
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-10-04T13:25:40+00:00
Next Scan 2024-10-18T13:25:40+00:00

Last Successful Scan

Scanned2024-09-19T13:24:48+00:00
URL https://vylustime.cz/robots.txt
Domain IPs 89.221.217.219
Response IP 89.221.217.219
Found Yes
Hash fd01a71f595824cefde831128af8b73e534d0d863f3bec3954a923475c9eff0c
SimHash 2a5fec52e251

Groups

*

Rule Path
Disallow /oou
Disallow /najit
Disallow /projit
Disallow /pridat
Disallow /diskuse
Disallow /cookies

seznambot
sklikbot

Rule Path
Disallow /oou
Disallow /najit
Disallow /projit
Disallow /pridat?*
Disallow /diskuse
Disallow /cookies

bingbot

Rule Path
Disallow /oou
Disallow /najit
Disallow /projit
Disallow /pridat
Disallow /diskuse
Disallow /cookies

Other Records

Field Value
crawl-delay 300

ahrefsbot
amazonbot
archive.org_bot
baiduspider
barkrowler
bleriot
blexbot
bubing
ccbot
cliqzbot
dataforseobot
dataprovider
desu
deusu
dnyzbot
dotbot
embedly
exabot
genieo
heritrix
ia_archiver
lcc
linguee
linkdexbot
linkpadbot
ltx71
mauibot
megaindex.ru
mj12bot
msnbot
netseer
qwantify
riddler
seekport
semrushbot
semrushbot-sa
seokicks
seokicks-robot
serpstatbot
smtbot
sogou
spbot
spiderling
surdotlybot
turnitinbot
uptimebot
velenpublicwebcrawler
xovibot
yandexbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://vylustime.cz/sitemap.xml

Warnings

  • 1 invalid line.
  • `request-rate` is not a known field.
  • `robot-version` is not a known field.