freedomen.ru
robots.txt
Robots Exclusion Standard data for freedomen.ru
Resource Scan
Scan Details
| Site Domain | freedomen.ru |
| Base Domain | freedomen.ru |
| Scan Status | Ok |
| Last Scan | 4/9/2025, 6:59:44 PM |
| Next Scan | 4/16/2025, 6:59:44 PM |
Last Scan
| Scanned | 4/9/2025, 6:59:44 PM |
| URL | https://freedomen.ru/robots.txt |
| Domain IPs | 104.21.8.234, 172.67.188.209, 2606:4700:3032::ac43:bcd1, 2606:4700:3036::6815:8ea |
| Response IP | 172.67.188.209 |
| Found | Yes |
| Hash | 15445f2d08495cbe950018b171075bffca9f7bf9969aba6b648d4a70f807155d |
| SimHash | 2202d8814521 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /reg/ |
| Disallow | /site/ |
| Disallow | /cat/ |
| Disallow | /away.php*$ |
| Disallow | /awaytop.php*$ |
| Disallow | /awaymid.php*$ |
| Disallow | /dedicated_collocation*$ |
| Disallow | /contact-form*$ |
| Disallow | /eurohost*$ |
| Disallow | /domain_en.shtml*$ |
| Disallow | /domen99rub.shtml*$ |
| Disallow | /recept.php*$ |
| Disallow | /dedicated_Germany.php |
| Disallow | /dedicated_Germany.php*$ |
| Disallow | /reg_domain_ru.php |
| Disallow | /reg_domain_ru.php*$ |
yandex
| Rule | Path |
|---|---|
| Disallow | /reg/ |
| Disallow | /site/ |
| Disallow | /cat/ |
| Disallow | /away.php*$ |
| Disallow | /awaytop.php*$ |
| Disallow | /awaymid.php*$ |
| Disallow | /dedicated_collocation*$ |
| Disallow | /contact-form*$ |
| Disallow | /eurohost*$ |
| Disallow | /domain_en.shtml*$ |
| Disallow | /domen99rub.shtml*$ |
| Disallow | /recept.php*$ |
| Disallow | /dedicated_Germany.php |
| Disallow | /dedicated_Germany.php*$ |
| Disallow | /reg_domain_ru.php |
| Disallow | /reg_domain_ru.php*$ |
| Disallow | /podbor.php.php |
| Disallow | /podbor.php.php*$ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 79 |
sputnikbot
| Rule | Path |
|---|---|
| Disallow | /reg/ |
| Disallow | /site/ |
| Disallow | /away.php*$ |
| Disallow | /dedicated_collocation*$ |
| Disallow | /eurohost*$ |
| Disallow | /domain_en.shtml*$ |
| Disallow | /domen99rub.shtml*$ |
| Disallow | /recept.php*$ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 99 |
exabot
| Rule | Path |
|---|---|
| Disallow | /reg/ |
| Disallow | /site/ |
| Disallow | /away.php*$ |
| Disallow | /dedicated_collocation*$ |
| Disallow | /eurohost*$ |
| Disallow | /domain_en.shtml*$ |
| Disallow | /domen99rub.shtml*$ |
| Disallow | /recept.php*$ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 99 |
slurp
| Rule | Path |
|---|---|
| Disallow | /reg/ |
| Disallow | /site/ |
| Disallow | /away.php*$ |
| Disallow | /dedicated_collocation*$ |
| Disallow | /eurohost*$ |
| Disallow | /domain_en.shtml*$ |
| Disallow | /domen99rub.shtml*$ |
| Disallow | /recept.php*$ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 79 |
yahoo
| Rule | Path |
|---|---|
| Disallow | /reg/ |
| Disallow | /site/ |
| Disallow | /away.php*$ |
| Disallow | /dedicated_collocation*$ |
| Disallow | /eurohost*$ |
| Disallow | /domain_en.shtml*$ |
| Disallow | /domen99rub.shtml*$ |
| Disallow | /recept.php*$ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 29 |
bingbot
| Rule | Path |
|---|---|
| Disallow | /reg/ |
| Disallow | /site/ |
| Disallow | /away.php*$ |
| Disallow | /dedicated_collocation*$ |
| Disallow | /eurohost*$ |
| Disallow | /domain_en.shtml*$ |
| Disallow | /domen99rub.shtml*$ |
| Disallow | /recept.php*$ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 27 |
Warnings
- `host` is not a known field.