computeruniverse.ru
robots.txt

Robots Exclusion Standard data for computeruniverse.ru

Resource Scan

Scan Details

Site Domain computeruniverse.ru
Base Domain computeruniverse.ru
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-08-30T00:58:06+00:00
Next Scan 2025-11-28T00:58:06+00:00

Last Successful Scan

Scanned2023-08-11T23:33:08+00:00
URL https://computeruniverse.ru/robots.txt
Domain IPs 104.21.6.99, 172.67.134.179, 2606:4700:3031::ac43:86b3, 2606:4700:3032::6815:663
Response IP 172.67.134.179
Found Yes
Hash 260e2055e29aec4991090edcd7cff2dff3cf4a12d93093878f188a400c3bb0de
SimHash 2a83c26153a1

Groups

e-societyrobot

Rule Path
Disallow /

fasterfox

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

cityreview

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

sp_auditbot

Rule Path
Disallow /

yandex

Rule Path
Allow /
Disallow /cart*
Disallow /community/orders/
Disallow /community/bonusaccount.asp
Disallow /community/bonuspoints/
Disallow /community/login.asp
Disallow /community/password.asp
Disallow /community/profile*
Disallow /cgi-bin/
Disallow /de/
Disallow /en/cgi-bin/
Disallow /en/trapit/
Disallow /help/
Disallow /nocookie.asp
Disallow /trapit/
Disallow /ru/
Allow /community/bonuspoints/default.asp
Allow /community/bonuspoints/praemien.asp

*

Rule Path
Disallow /
Allow /default.asp
Allow /$
Allow /css/
Allow /lib/
Allow /images/
Allow /images2/
Allow /group/kabylake/kaby-lake-intel.asp

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1