obrazovanie.by
robots.txt

Robots Exclusion Standard data for obrazovanie.by

Resource Scan

Scan Details

Site Domain obrazovanie.by
Base Domain obrazovanie.by
Scan Status Ok
Last Scan2024-10-05T07:22:29+00:00
Next Scan 2024-10-12T07:22:29+00:00

Last Scan

Scanned2024-10-05T07:22:29+00:00
URL https://obrazovanie.by/robots.txt
Domain IPs 2a0a:7d80:1:7::103:f2, 93.125.99.85
Response IP 93.125.99.85
Found Yes
Hash c75849063b19068169931c46c16644c0044e6da5e9ba4192c6b207d336b8ba57
SimHash 25113073d799

Groups

*

Rule Path
Disallow /cgi-bin
Disallow /?
Disallow /wp-
Disallow *?s=
Disallow *%26s%3D
Disallow /search/
Disallow /author/
Disallow /users/
Disallow */trackback
Disallow */feed
Disallow */rss
Disallow */embed
Disallow /xmlrpc.php
Disallow *utm*%3D
Disallow *openstat%3D
Allow /*/*.js
Allow /*/*.css
Allow /wp-*.png
Allow /wp-*.jpg
Allow /wp-*.jpeg
Allow /wp-*.webm
Allow /wp-*.gif
Allow /wp-*.ttf
Allow /wp-*.woff
Allow /wp-admin/admin-ajax.ph

bingbot

Rule Path
Disallow /

bingbot/2.0

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

converacrawler

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

megaindexauditbot

Rule Path
Disallow /

vedma

Rule Path
Disallow /

lcc

Rule Path
Disallow /

ntentbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://obrazovanie.by/sitemap_index.xml

Warnings

  • 2 invalid lines.
  • `host` is not a known field.