Log In Sign Up

civ-blog.ru
robots.txt

Robots Exclusion Standard data for civ-blog.ru

Resource Scan

Scan Details

Site Domain civ-blog.ru
Base Domain civ-blog.ru
Scan Status Ok
Last Scan2025-09-18T14:26:12+00:00
Next Scan 2025-09-25T14:26:12+00:00

Last Scan

Scanned2025-09-18T14:26:12+00:00
URL https://civ-blog.ru/robots.txt
Domain IPs 77.222.57.82
Response IP 77.222.57.82
Found Yes
Hash 212439c085ae623303b03c851510f45ca7669e76b9907bf9df2e99a4680a54f8
SimHash 6f78de22e255

Groups

yandex

Rule Path
Disallow /cgi-bin/
Disallow /wp-login.php
Disallow /wp-register.php
Disallow /wp-includes/
Disallow /xmlrpc.php
Disallow /wp-admin
Disallow /wp-content/plugins
Disallow /wp-content/themes
Disallow /wp-content/cache
Disallow /tag/
Disallow /page/
Disallow /category/
Disallow /author/
Disallow */page
Disallow /trackback/
Disallow /wp-feed
Disallow /comments/
Disallow */trackback/
Disallow */comments/
Disallow /?feed=
Disallow /?s=
Disallow /*?*
Disallow /*?

Other Records

Field Value
crawl-delay 10

slurp

Rule Path
Disallow /

*

Rule Path
Disallow /cgi-bin/
Disallow /wp-login.php
Disallow /wp-register.php
Disallow /wp-includes/
Disallow /xmlrpc.php
Disallow /wp-admin
Disallow /wp-content/plugins
Disallow /wp-content/themes
Disallow /wp-content/cache
Disallow /tag/
Disallow /page/
Disallow /category/
Disallow /author/
Disallow */page
Disallow /trackback/
Disallow /wp-feed
Disallow /comments/
Disallow */trackback/
Disallow */comments/
Disallow /?feed=
Disallow /?s=
Disallow /*?*
Disallow /*?

Other Records

Field Value
crawl-delay 10
Back to top

Other Records

Field Value
sitemap https://www.civ-blog.ru/sitemap-2.xml
sitemap https://www.civ-blog.ru/sitemap-2.xml
Back to top

Warnings

  • `host` is not a known field.
Back to top