Log In Sign Up

вести-касторное.рф
robots.txt

Robots Exclusion Standard data for вести-касторное.рф

Resource Scan

Scan Details

Site Domain вести-касторное.рф
Base Domain вести-касторное.рф
Scan Status Ok
Last Scan2024-04-28T17:38:55+00:00
Next Scan 2024-05-28T17:38:55+00:00

Last Scan

Scanned2024-04-28T17:38:55+00:00
URL https://вести-касторное.рф/robots.txt
Redirect https://xn----8sbemcppzibugejf.xn--p1ai/robots.txt
Domain IPs 104.21.81.241, 172.67.192.30, 2606:4700:3034::ac43:c01e, 2606:4700:3035::6815:51f1
Response IP 104.21.81.241
Found Yes
Hash 978380b39e5107bcc2bb794e2fead0e0a98b98a7ec57d814adbe30aecd2bf36f
SimHash 9f3e1de8037d

Groups

*

Rule Path
Disallow /administrator/
Disallow /bin/
Disallow /cache/
Disallow /cli/
Disallow /components/
Disallow /includes/
Disallow /installation/
Disallow /language/
Disallow /libraries/
Disallow /logs/
Disallow /media/
Disallow /modules/
Disallow /plugins/
Disallow /templates/
Disallow /tmp/
Disallow /blog_calendar/
Disallow /avtorizatsiya.html*
Disallow /component/
Disallow /content/
Disallow /index/
Disallow /*start%3D*
Disallow /*catid*
Disallow /*Itemid*
Disallow /*option*
Disallow /*com_conten*
Disallow /*article*

Other Records

Field Value
crawl-delay 59

yandexnews

Rule Path
Allow /rss-ya

Other Records

Field Value
crawl-delay 2
Back to top

Comments

  • For more information about the robots.txt standard, see:
  • http://www.robotstxt.org/orig.html
  • For syntax checking, see:
  • http://tool.motoricerca.info/robots-checker.phtml
Back to top

Warnings

  • `host` is not a known field.
Back to top