Log In Sign Up

медитация.рф
robots.txt

Robots Exclusion Standard data for медитация.рф

Resource Scan

Scan Details

Site Domain медитация.рф
Base Domain медитация.рф
Scan Status Ok
Last Scan2024-09-27T12:09:27+00:00
Next Scan 2024-10-04T12:09:27+00:00

Last Scan

Scanned2024-09-27T12:09:27+00:00
URL https://медитация.рф/robots.txt
Redirect https://xn--80ahcnbt9b7a1f.xn--p1ai/robots.txt
Domain IPs 2a00:f940:2:2:1:1:0:122, 37.140.192.211
Response IP 37.140.192.211
Found Yes
Hash c651c1954b9a8dd920dc8679e468e4c238b8459ff623f47a7688c3735f118b13
SimHash e21ebe738b00

Groups

*

Rule Path
Allow /*.js*
Allow /*.css*
Allow /*.png*
Allow /*.jpg*
Allow /*.gif*
Disallow /administrator/
Disallow /cache/
Disallow /cli/
Disallow /includes/
Disallow /installation/
Disallow /language/
Disallow /libraries/
Disallow /logs/
Disallow /tmp/
Disallow /component/jcomments/captcha/
Disallow /component/tags/tag/
Disallow /%D1%80%D0%B0%D1%81%D1%81%D1%8B%D0%BB%D0%BA%D0%B8/url/
Disallow /?showtopic
Disallow /?topic=
Disallow /?board
Disallow /component/acymailing/
Disallow /*?start=
Disallow /*start
Disallow /*searchword
Disallow /?author=
Disallow /?xxnew
Disallow /?v=
Disallow /?auid=
Disallow /?act=
Disallow /?acm=
Disallow /?app=
Disallow /index.php?subid=
Disallow /index.php?option=
Disallow /?tp=
Disallow /?controller=
Disallow /?option
Disallow /?q=
Disallow /?do=
Disallow /?xxxxxxxxxxxx_loads
Disallow /?action
Disallow /?s=
Disallow /?ac=
Disallow /?controller
Disallow /?showuser
Disallow /?template
Disallow /forum/
Disallow /?showforum
Disallow /component/banners/
Disallow /component/phocagallery/category
Disallow /component/users/
Disallow /14-
Disallow /118-
Disallow /54-
Disallow /102-
Disallow /?page=
Disallow /2-
Disallow /components/com_jdownloads/assets
Disallow /ql?view
Disallow /?%3F%3F%3F%3F%3F%3F%3F%3F
Disallow /*?utm_source
Disallow /*?task

yandex

Rule Path
Disallow
Allow /*.js*
Allow /*.css*
Allow /*.png*
Allow /*.jpg*
Allow /*.gif*
Disallow /administrator/
Disallow /cache/
Disallow /cli/
Disallow /includes/
Disallow /installation/
Disallow /language/
Disallow /libraries/
Disallow /logs/
Disallow /tmp/
Disallow /component/jcomments/captcha/
Disallow /component/tags/tag/
Disallow /%D1%80%D0%B0%D1%81%D1%81%D1%8B%D0%BB%D0%BA%D0%B8/url/
Disallow /?showtopic
Disallow /?topic=
Disallow /?board
Disallow /component/acymailing/
Disallow /*?start=
Disallow /*start
Disallow /*searchword
Disallow /?author=
Disallow /?xxnew
Disallow /?v=
Disallow /?auid=
Disallow /?act=
Disallow /?acm=
Disallow /?app=
Disallow /index.php?subid=
Disallow /index.php?option=
Disallow /?tp=
Disallow /?controller=
Disallow /?option
Disallow /?q=
Disallow /?do=
Disallow /?xxxxxxxxxxxx_loads
Disallow /?action
Disallow /?s=
Disallow /?ac=
Disallow /?controller
Disallow /?showuser
Disallow /?template
Disallow /forum/
Disallow /?showforum
Disallow /component/banners/
Disallow /component/phocagallery/category
Disallow /component/users/
Disallow /14-
Disallow /118-
Disallow /54-
Disallow /102-
Disallow /?page=
Disallow /2-
Disallow /components/com_jdownloads/assets
Disallow /ql?view
Disallow /?%3F%3F%3F%3F%3F%3F%3F%3F
Disallow /*?utm_source
Disallow /*?task

wget

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

serpstatbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 30

semrushbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 30

okhttp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30
crawl-delay 10
Back to top

Other Records

Field Value
sitemap https://xn--80ahcnbt9b7a1f.xn--p1ai/index.php?option=com_jmap&view=sitemap&format=xml
sitemap https://xn--80ahcnbt9b7a1f.xn--p1ai/sitemapindex_xml.xml
Back to top

Comments

  • If the Joomla site is installed within a folder such as at
  • e.g. www.example.com/joomla/ the robots.txt file MUST be
  • moved to the site root at e.g. www.example.com/robots.txt
  • AND the joomla folder name MUST be prefixed to the disallowed
  • path, e.g. the Disallow rule for the /administrator/ folder
  • MUST be changed to read Disallow: /joomla/administrator/
  • For more information about the robots.txt standard, see:
  • http://www.robotstxt.org/orig.html
  • For syntax checking, see:
  • http://www.sxw.org.uk/computing/robots/check.html
  • Deny SEMrushBot from crawling your site for web graph of links, add:
  • Deny SEMrushBot from crawling your site for different SEO and technical issues, add:
  • User-agent: SemrushBot-SA
  • Disallow: /
  • Crawl-delay: 30
  • и других ботов из htaccess
  • User-agent: BlackWidow
  • Disallow: /
  • Crawl-delay: 30
  • User-agent: AhrefsBot
  • Disallow: /
  • Crawl-delay: 30
  • User-agent: BLEXBot
  • Disallow: /
  • Crawl-delay: 30
  • User-agent: MBCrawler
  • Disallow: /
  • Crawl-delay: 30
  • User-agent: YaK
  • Disallow: /
  • Crawl-delay: 30
  • User-agent: Megaindex
  • Disallow: /
  • Crawl-delay: 30
  • User-agent: MJ12bot
  • Disallow: /
  • Crawl-delay: 30
  • User-agent: cloudfind
  • Disallow: /
  • Crawl-delay: 30
  • User-agent: CriteoBot
  • Disallow: /
  • Crawl-delay: 30
  • User-agent: GetIntent Crawler
  • Disallow: /
  • Crawl-delay: 30
  • User-agent: SafeDNSBot
  • Disallow: /
  • Crawl-delay: 30
  • User-agent: SeopultContentAnalyzer
  • Disallow: /
  • Crawl-delay: 30
  • User-agent: LinkpadBot
  • Disallow: /
  • Crawl-delay: 30
  • User-agent: Slurp
  • Disallow: /
  • Crawl-delay: 30
  • User-agent: DataForSeoBot
  • Disallow: /
  • Crawl-delay: 30
  • User-agent: Rome Client
  • Disallow: /
  • Crawl-delay: 30
  • User-agent: Scrapy
  • Disallow: /
  • Crawl-delay: 30
  • User-agent: FlipboardRSS
  • Disallow: /
  • Crawl-delay: 30
  • User-agent: FlipboardProxy
  • Disallow: /
  • Crawl-delay: 30
  • User-agent: ZoominfoBot
  • Disallow: /
  • Crawl-delay: 30
  • User-agent: SeznamBot
  • Disallow: /
  • Crawl-delay: 30
  • User-agent: Seekport Crawler
  • Disallow: /
  • Crawl-delay: 30
  • User-agent: SeekportBot
  • Disallow: /
  • Crawl-delay: 30
  • JSitemap entries
Back to top

Warnings

  • `clean-param` is not a known field.
  • `host` is not a known field.
Back to top