kuban.info
robots.txt

Robots Exclusion Standard data for kuban.info

Resource Scan

Scan Details

Site Domain kuban.info
Base Domain kuban.info
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2025-08-22T19:20:49+00:00
Next Scan 2025-10-21T19:20:49+00:00

Last Successful Scan

Scanned2025-06-23T20:15:48+00:00
URL https://kuban.info/robots.txt
Domain IPs 2a00:f940:2:2:1:4:0:117, 37.140.192.138
Response IP 37.140.192.138
Found Yes
Hash 1bc824112e52064311883091f1927b51c3c9250aa598d71951e3af75e50de69c
SimHash 754dd87b8033

Groups

mail.ru

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

petalbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

*

Rule Path
Disallow /engine/go.php
Disallow /engine/download.php
Disallow /user/
Disallow /newposts/
Disallow /statistics.html
Disallow /*subaction%3Duserinfo
Disallow /*subaction%3Dnewposts
Disallow /*do%3Dlastcomments
Disallow /*action%3Dmobile
Disallow /*do%3Dfeedback
Disallow /*do%3Dregister
Disallow /*do%3Dlostpassword
Disallow /*do%3Daddnews
Disallow /*do%3Dstats
Disallow /*do%3Dpm
Disallow /*do%3Dsearch
Disallow /uploads/
Disallow /favorites/
Disallow /tags/
Disallow /lastnews/
Disallow /rules.html
Disallow /2012/
Disallow /2013/
Disallow /2014/
Disallow /2015/
Disallow /2016/
Disallow /2017/

Other Records

Field Value
sitemap https://kuban.info/sitemap.xml

Warnings

  • `host` is not a known field.