Log In Sign Up

43edu.ru
robots.txt

Robots Exclusion Standard data for 43edu.ru

Resource Scan

Scan Details

Site Domain 43edu.ru
Base Domain 43edu.ru
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-05-12T02:33:00+00:00
Next Scan 2024-07-11T02:33:00+00:00

Last Successful Scan

Scanned2024-02-20T02:24:22+00:00
URL https://43edu.ru/robots.txt
Domain IPs 188.225.30.159
Response IP 188.225.30.159
Found Yes
Hash fb95b27ed21e3565869b27a0eca8ef59f92f51c378f907279022e93bc001079a
SimHash 44765c66d617

Groups

yandex

Rule Path
Allow /
Disallow /upload/
Disallow /bitrix/

Other Records

Field Value
crawl-delay 2

yandexbot

Rule Path
Allow /
Disallow /upload/
Disallow /bitrix/

Other Records

Field Value
crawl-delay 2

exabot

Rule Path
Disallow /

nkcki

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

crawl

Rule Path
Disallow /

bot

Rule Path
Disallow /
Back to top

Comments

  • Sitemap: https://43edu.ru/map/sitemap.xml
  • User-Agent: *
  • Disallow: /
  • Crawl-delay: 600
  • Request-rate: 1/10
  • Visit-time: 0400-0730
  • Sitemap: https://43edu.ru/map/sitemap.xml
  • User-agent: Googlebot
  • Allow: /news
  • Allow: /ministry
  • Allow: /business
  • Allow: /contacts
  • Allow: /regulations
  • Disallow: /
  • Host: 43edu.ru
  • Crawl-delay: 10
  • Request-rate: 1/10
  • Visit-time: 0400-0730
  • Sitemap: https://43edu.ru/map/sitemap.xml
  • User-Agent: *
  • Sitemap: https://43edu.ru/map/sitemap.xml
Back to top

Warnings

  • `clean-param` is not a known field.
  • `host` is not a known field.
Back to top