smolmama.ru
robots.txt

Robots Exclusion Standard data for smolmama.ru

Resource Scan

Scan Details

Site Domain smolmama.ru
Base Domain smolmama.ru
Scan Status Ok
Last Scan2024-09-17T09:08:51+00:00
Next Scan 2024-09-24T09:08:51+00:00

Last Scan

Scanned2024-09-17T09:08:51+00:00
URL http://smolmama.ru/robots.txt
Redirect http://www.smolmama.ru/robots.txt
Redirect Domain www.smolmama.ru
Redirect Base smolmama.ru
Domain IPs 2a00:15f8:a000:5:1:11:6:c4a4, 2a00:15f8:a000:5:1:12:6:c4a4, 2a00:15f8:a000:5:1:13:6:c4a4, 2a00:15f8:a000:5:1:14:6:c4a4, 90.156.201.114, 90.156.201.26, 90.156.201.44, 90.156.201.82
Redirect IPs 2a00:15f8:a000:5:1:11:6:c4a4, 2a00:15f8:a000:5:1:12:6:c4a4, 2a00:15f8:a000:5:1:13:6:c4a4, 2a00:15f8:a000:5:1:14:6:c4a4, 90.156.201.114, 90.156.201.26, 90.156.201.44, 90.156.201.82
Response IP 90.156.201.114
Found Yes
Hash 3b3dc3850273b4c8a2c7023b0046fd050cf36c3e73cb5a7444684a1978ea3c26
SimHash 6f409e73cbbd

Groups

*

Rule Path
Allow /forum/*sitemap
Allow /forum/*arcade
Allow /forum/*rss
Disallow /forum/attachments/
Disallow /forum/avatars/
Disallow /forum/Packages/
Disallow /forum/Smileys/
Disallow /forum/Sources/
Disallow /forum/Themes/
Disallow /forum/Games/
Disallow /forum/*.msg
Disallow /forum/*.new
Disallow /forum/*sort
Disallow /forum/*topicseen
Disallow /forum/*wap
Disallow /forum/*imode
Disallow /forum/*action

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 100

twiceler

Rule Path
Disallow /

w3c-checklink

Rule Path
Disallow /

msnbot
msnbot-media

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

ahrefsbot

Rule Path
Disallow /

Warnings

  • `host` is not a known field.