Log In Sign Up

e-usti.cz
robots.txt

Robots Exclusion Standard data for e-usti.cz

Resource Scan

Scan Details

Site Domain e-usti.cz
Base Domain e-usti.cz
Scan Status Ok
Last Scan2024-11-16T05:38:14+00:00
Next Scan 2024-11-23T05:38:14+00:00

Last Scan

Scanned2024-11-16T05:38:14+00:00
URL https://e-usti.cz/robots.txt
Redirect http://www.e-usti.cz/robots.txt
Redirect Domain www.e-usti.cz
Redirect Base e-usti.cz
Domain IPs 81.0.226.126
Redirect IPs 81.0.226.126
Response IP 81.0.226.126
Found Yes
Hash eac9fd4eae9f348fe769d924be4c206753ae5403098ded56625c4cd41a540af6
SimHash 600e346ac3e2

Groups

*

Rule Path
Allow /*.js*
Allow /*.css*
Allow /*.png*
Allow /*.jpg*
Allow /*.gif*
Allow /*.webp*
Disallow /administrator/
Disallow /bin/
Disallow /cache/
Disallow /cli/
Disallow /includes/
Disallow /installation/
Disallow /language/
Disallow /layouts/
Disallow /libraries/
Disallow /logs/
Disallow /tmp/

ahrefsbot

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

bingbot

Rule Path
Disallow /

bingbot/2.0

Rule Path
Disallow /

adbeat_bot

Rule Path
Disallow /

agentlinkspammer

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

ahrefsbot/4.0

Rule Path
Disallow /

aihitbot

Rule Path
Disallow /

aihitbot/1.0

Rule Path
Disallow /

aihitbot/1.1

Rule Path
Disallow /

acoon

Rule Path
Disallow /

arachmo

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduspider+

Rule Path
Disallow /

baiduspider+(+http://www.baidu.com/search/spider.htm)

Rule Path
Disallow /

baiduspider/2.0;+http://www.baidu.com/search/spider.html

Rule Path
Disallow /

baiduspider/2.0

Rule Path
Disallow /

mozilla/5.0(compatible; baiduspider/2.0; +http://www.baidu.com/search/spider.html)

Rule Path
Disallow /

careerbot

Rule Path
Disallow /

comodospider/nutch-1.2

Rule Path
Disallow /

easouspider

Rule Path
Disallow /

exabot/3.0

Rule Path
Disallow /

exalead

Rule Path
Disallow /

exalead crawler

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

ezooms/1.0

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

mozilla/5.0 (compatible; ezooms/1.0; ezooms.bot[at]gmail[dot]com)

Rule Path
Disallow /

findlinks/2.6

Rule Path
Disallow /

java/1.6.0_04

Rule Path
Disallow /

jikespider

Rule Path
Disallow /

kaloogabot

Rule Path
Disallow /

mail.ru_bot/2.0

Rule Path
Disallow /

mail.ru

Rule Path
Disallow /

mail.ru

Rule Path
Disallow /

mail.ru_bot/2.0; +http://go.mail.ru/help/robots

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

mj12bot/v1.4.3

Rule Path
Disallow /

moget

Rule Path
Disallow /

ichiro

Rule Path
Disallow /

ichiro 3.0

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

netcraftsurveyagent/1.0

Rule Path
Disallow /

openwebindex/nutch-1.6

Rule Path
Disallow /

openwebindex

Rule Path
Disallow /

panoptastudybot

Rule Path
Disallow /

checks.panopta.com

Rule Path
Disallow /

psbot

Rule Path
Disallow /

plukkie

Rule Path
Disallow /

sistrixcrawler

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

sistrix crawler

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

sosospider+

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

sosospider/2.0

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

wada.vn

Rule Path
Disallow /

wada.vn vietnamese search

Rule Path
Disallow /

wada.vn vietnamese search/2.1

Rule Path
Disallow /

yandex

Rule Path
Disallow /

yandex/1.01.001

Rule Path
Disallow /

yandexbot/3.0-mirrordetector

Rule Path
Disallow /

yandeximages/3.0

Rule Path
Disallow /

yandexsomething/1.

Rule Path
Disallow /

yandex.com

Rule Path
Disallow /

yandexbot/3.0

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

youdaobot/1.0

Rule Path
Disallow /

youdaobot/1.0

Rule Path
Disallow /

zao

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

ahrefsbot/5.2

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

sentibot

Rule Path
Disallow /

msnbot-media

Rule Path
Disallow /

msnbot-media/1.1

Rule Path
Disallow /

mediatoolkitbot

Rule Path
Disallow /

monitorabot

Rule Path
Disallow /

monitorabot/1.0

Rule Path
Disallow /

monitorabot

Rule Path
Disallow /
Back to top

Comments

  • If the Joomla site is installed within a folder
  • eg www.example.com/joomla/ then the robots.txt file
  • MUST be moved to the site root
  • eg www.example.com/robots.txt
  • AND the joomla folder name MUST be prefixed to all of the
  • paths.
  • eg the Disallow rule for the /administrator/ folder MUST
  • be changed to read
  • Disallow: /joomla/administrator/
  • For more information about the robots.txt standard, see:
  • http://www.robotstxt.org/orig.html
  • For syntax checking, see:
  • http://tool.motoricerca.info/robots-checker.phtml
  • Adbeat ads
  • AgentLinkSpammer
  • AhrefsBot ads
  • aiHitBot Ukraine or Russia
  • Acoon Germany
  • Arachmo Japan
  • Baiduspider China and Japan
  • careerbot Germany
  • COMODOSpider/Nutch-1.2 United Kingdom
  • EasouSpider - China
  • Exabot/3.0 - France proxy scraper
  • Exalead proxy scraper France
  • Ezooms and dotbot
  • findlinks/2.6 Germany http://wortschatz.uni-leipzig.de/findlinks
  • Java/1.6.0_04
  • JikeSpider China
  • KaloogaBot Netherlands contextual advertising
  • Mail.RU_Bot/2.0 Russia
  • Mail.RU Russia
  • Mail.Ru Russia
  • MJ12bot United Kingdom
  • MJ12bot/v1.4.3 United Kingdon
  • Ichiro Japan
  • Ichiro 3.0 Japan
  • NetcraftSurveyAgent/1.0
  • OpenWebIndex/Nutch-1.6 Germany
  • panoptaStudyBot checks.panopta.com monitor
  • panoptaStudyBot checks.panopta.com monitor
  • picsearch Sweden searches for pictures
  • plukkie Dutch (botje.nl)/Belgium (botje.be)/France (botje.fr)/United Kingdom (botje.co.uk) search engine
  • SistrixCrawler Germany DE
  • Sogou
  • Sosospider - China http://help.soso.com/webspider.htm
  • Sosospider - China
  • Sosospider/2.0 - China may not obey robots.txt
  • 360Spider China
  • SurveyBot
  • Wada.vn Vietnamese Search/2.1
  • Yandex
  • YisouSpider China
  • YoudaoBot/1.0 China
  • YoudaoBot China
  • Zao - Japan
Back to top

Warnings

  • 8 invalid lines.
  • `robot-version` is not a known field.
Back to top