Log In Sign Up

6lib.ru
robots.txt

Robots Exclusion Standard data for 6lib.ru

Resource Scan

Scan Details

Site Domain 6lib.ru
Base Domain 6lib.ru
Scan Status Ok
Last Scan2024-11-02T07:27:21+00:00
Next Scan 2024-11-09T07:27:21+00:00

Last Scan

Scanned2024-11-02T07:27:21+00:00
URL https://6lib.ru/robots.txt
Domain IPs 213.239.215.117
Response IP 213.239.215.117
Found Yes
Hash 279e31c5c6d41aa711be1a60126992b9aa248576c41d74b53c02457507a72f9e
SimHash 674090e0cd31

Groups

*

Rule Path
Disallow /download/
Disallow /*?download=*
Disallow /*?page=*
Disallow *?search=*
Disallow *?setorder=*
Disallow /.git/

mediapartners-google

Rule Path
Allow /download/

yandex

Rule Path
Disallow /download/
Disallow /qrcode/
Disallow /*?download=*
Disallow /*?page=*
Disallow /.page-*
Disallow *?search=*
Disallow *?setorder=*
Disallow /.git/

Other Records

Field Value
crawl-delay 5
Back to top

Other Records

Field Value
sitemap https://www.6lib.ru/sitemap.xml
Back to top

Warnings

  • `clean-param` is not a known field.
  • `host` is not a known field.
Back to top