Log In Sign Up

mr-software.cz
robots.txt

Robots Exclusion Standard data for mr-software.cz

Resource Scan

Scan Details

Site Domain mr-software.cz
Base Domain mr-software.cz
Scan Status Ok
Last Scan2026-01-21T18:29:43+00:00
Next Scan 2026-02-20T18:29:43+00:00

Last Scan

Scanned2026-01-21T18:29:43+00:00
URL https://mr-software.cz/robots.txt
Domain IPs 81.2.195.60
Response IP 81.2.195.60
Found Yes
Hash 5a95ac414f409fe409ace82192d9cb7cb9cfecb2c85b13466c0e42b191f01831
SimHash 38983941cdd0

Groups

*

Rule Path
Disallow /aspnet_client/
Disallow /admin/
Disallow /bin/
Disallow /log/
Disallow /data/
Disallow /download/
Disallow /downloads/
Disallow /test/
Disallow /gallery/

blexbot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

yandex

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

semrushbot-bm

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

applebot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

youbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

piplbot

Rule Path
Disallow /
Back to top

Comments

  • Open Link Profiler
  • Mozilla/5.0+(compatible;+spbot/4.4.2;++http://OpenLinkProfiler.org/bot+)
  • http://OpenLinkProfiler.org/bot
  • http://mj12bot.com - MAJESTIC
  • Crawl-Delay should be an integer number and it signifies number of seconds of wait between requests.
  • Crawl-Delay: 5
  • https://ahrefs.com/robot - Mozilla/5.0+(compatible;+AhrefsBot/5.2;++http://ahrefs.com/robot/)
  • bingbot
  • blocks access to whole site for All Yandex bots like YandexBot, YandexDirect, YandexDirectDyn, YandexMedia, YandexImages ....
  • block SEMrushBot
  • To block SEMrushBot from crawling your site for a webgraph of links:
  • To block SEMrushBot from crawling your site for different SEO and technical issues:
  • To block SEMrushBot from crawling your site for Backlink Audit tool:
  • To block SEMrushBot from crawling your site for On Page SEO Checker tool and similar tools:
  • To block SEMrushBot from checking URLs your site for SWA tool:
  • To block SEMrushBot from crawling your site for Content Analyzer and Post Tracking tools:
  • To block SEMrushBot from crawling your site for Brand Monitoring:
  • Here are AI bots I’m blocking using robots.txt
Back to top