abitu.ru
robots.txt

Robots Exclusion Standard data for abitu.ru

Resource Scan

Scan Details

Site Domain abitu.ru
Base Domain abitu.ru
Scan Status Ok
Last Scan2026-03-06T00:19:14+00:00
Next Scan 2026-03-13T00:19:14+00:00

Last Scan

Scanned2026-03-06T00:19:14+00:00
URL https://abitu.ru/robots.txt
Domain IPs 2a00:f940:2:2:1:4:0:113, 31.31.198.249
Response IP 31.31.198.249
Found Yes
Hash 34b61e233b997944fd2817e884ebf8fc5ddde0e689119b4efaf4c7d89086cf84
SimHash 4d0df8708221

Groups

mail.ru

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

petalbot

Rule Path
Disallow /

*

Rule Path
Disallow /engine/go.php
Disallow /engine/download.php
Disallow /user/
Disallow /page/
Disallow /tags/
Disallow /newposts/
Disallow /statistics.html
Disallow /*subaction%3Duserinfo
Disallow /*subaction%3Dnewposts
Disallow /*do%3Dlastcomments
Disallow /*do%3Dfeedback
Disallow /*do%3Dregister
Disallow /*do%3Dlostpassword
Disallow /*do%3Daddnews
Disallow /*do%3Dstats
Disallow /*do%3Dpm
Disallow /*do%3Dsearch
Disallow /*do%3Ddownload
Disallow /*do%3Dgo
Disallow /favorites/
Disallow /lastnews/
Disallow /2018/
Disallow /2019/
Disallow /2020/
Disallow /2021/

Other Records

Field Value
sitemap https://abitu.ru/sitemap.xml

Warnings

  • `host` is not a known field.