Log In Sign Up

vidilab.com
robots.txt

Robots Exclusion Standard data for vidilab.com

Resource Scan

Scan Details

Site Domain vidilab.com
Base Domain vidilab.com
Scan Status Ok
Last Scan2025-12-04T06:48:46+00:00
Next Scan 2026-01-03T06:48:46+00:00

Last Scan

Scanned2025-12-04T06:48:46+00:00
URL https://vidilab.com/robots.txt
Domain IPs 185.62.75.144
Response IP 185.62.75.144
Found Yes
Hash 3f774b21f4a6cbd4c62f0eb9fa3c29feac9c20b63ed541460abd47f79a60764a
SimHash fb1c1551c7e8

Groups

amazonbot

Rule Path
Allow *

claudebot

Rule Path
Allow *

bingbot

Rule Path
Allow *

yandex

Rule Path
Allow *

baiduspider

Rule Path
Allow *

sogou

Rule Path
Allow *

mj12bot

Rule Path
Allow *

googlebot

Rule Path
Allow *

moodlebot

Rule Path
Allow *

mediatoolkitbot

Rule Path
Allow *

applebot

Rule Path
Allow *

trendictionbot

Rule Path
Allow *

duckduckbot

Rule Path
Allow *

neticle crawler

Rule Path
Allow *

owler

Rule Path
Allow *

pressenginebot

Rule Path
Allow *

sentibot

Rule Path
Allow *

fiperbot

Rule Path
Allow *

bytespider

Rule Path
Allow *

rssingbot

Rule Path
Allow *

newspaper

Rule Path
Allow *

twitterbot

Rule Path
Allow *

barkrowler

Rule Path
Allow *

meta-externalagent

Rule Path
Allow *

ahrefsbot

Rule Path
Allow *

archivebot

Rule Path
Allow *

semrushbot

Rule Path
Allow *

petalbot

Rule Path
Allow *

blexbot

Rule Path
Allow *

dotbot

Rule Path
Allow *

gptbot

Rule Path
Allow *

amazonbot

Rule Path
Allow *

siteauditbot

Rule Path
Allow *

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

semrushbot-bm

Rule Path
Disallow /

splitsignalbot

Rule Path
Disallow /

semrushbot-coub

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

*

Rule Path
Disallow /administrator/
Disallow /bin/
Disallow /cache/
Disallow /cli/
Disallow /components/
Disallow /includes/
Disallow /installation/
Disallow /language/
Disallow /layouts/
Disallow /libraries/
Disallow /logs/
Disallow /modules/
Disallow /plugins/
Disallow /tmp/
Back to top

Other Records

Field Value
sitemap https://vidilab.com/index.php?option=com_xmap&view=xml&tmpl=component&id=1
sitemap https://vidilab.com/index.php?option=com_xmap&view=xml
Back to top

Comments

  • If the Joomla site is installed within a folder such as at
  • e.g. www.example.com/joomla/ the robots.txt file MUST be
  • moved to the site root at e.g. www.example.com/robots.txt
  • AND the joomla folder name MUST be prefixed to the disallowed
  • path, e.g. the Disallow rule for the /administrator/ folder
  • MUST be changed to read Disallow: /joomla/administrator/
  • For more information about the robots.txt standard, see:
  • http://www.robotstxt.org/orig.html
  • For syntax checking, see:
  • http://tool.motoricerca.info/robots-checker.phtml
  • Crawl-delay: 20
  • ClaudeBot
  • Crawl-delay: 50
  • Bingbot (Microsoft/Bing)
  • Crawl-delay: 5
  • Yandex (Rusija)
  • Crawl-delay: 50
  • Baidu (Kina)
  • Crawl-delay: 50
  • Sogou (Kina)
  • Crawl-delay: 50
  • 360Spider (Kina)
  • Crawl-delay: 50
  • MJ12bot (Web istraživanje)
  • Crawl-delay: 50
  • Googlebot
  • Crawl-delay: 2
  • MoodleBot
  • Crawl-delay: 20
  • Mediatoolkitbot
  • Crawl-delay: 20
  • Applebot
  • Crawl-delay: 10
  • trendictionbot
  • Crawl-delay: 20
  • DuckDuckBot
  • Crawl-delay: 5
  • Neticle Crawler
  • Crawl-delay: 20
  • Owler
  • Crawl-delay: 30
  • PressEngineBot
  • Crawl-delay: 20
  • SentiBot
  • Crawl-delay: 20
  • fiperbot
  • Crawl-delay: 30
  • Bytespider
  • Crawl-delay: 20
  • RSSingBot
  • Crawl-delay: 30
  • newspaper
  • Crawl-delay: 20
  • Twitterbot
  • Crawl-delay: 10
  • Barkrowler
  • Crawl-delay: 20
  • meta-externalagent
  • Crawl-delay: 20
  • Crawl-delay: 30
  • Crawl-delay: 30
  • Crawl-delay: 30
  • Crawl-delay: 30
  • Crawl-delay: 30
  • Crawl-delay: 30
  • Crawl-delay: 20
  • Crawl-delay: 20
  • Crawl-delay: 20
Back to top

Warnings

  • 2 invalid lines.
Back to top