list-org.com
robots.txt
Robots Exclusion Standard data for list-org.com
Resource Scan
Scan Details
| Site Domain | list-org.com |
| Base Domain | list-org.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2024-04-11T04:00:51+00:00 |
| Next Scan | 2024-06-10T04:00:51+00:00 |
Last Successful Scan
| Scanned | 2024-02-12T00:42:01+00:00 |
| URL | https://list-org.com/robots.txt |
| Redirect | https://www.list-org.com/robots.txt |
| Redirect Domain | www.list-org.com |
| Redirect Base | list-org.com |
| Domain IPs | 89.223.31.76 |
| Redirect IPs | 89.223.31.76 |
| Response IP | 89.223.31.76 |
| Found | Yes |
| Hash | 0f3084b25cade5d144770d7de8acbf7aa9fbd72e68eb4927f5adf764d3b66de6 |
| SimHash | 06ef5a4246da |
Groups
ahrefsbot
slurp
aport
teoma
proximic
scooter
ia_archiver
domaincrawler
lycos
webalta
mj12bot
grapeshot
baiduspider
baiduspider-video
sogou spider
youdaobot
naverbot
yeti
moget
ichiro
hybridbot
semrushbot
semrushbot-sa
smtbot
blexbot
dotbot
getintent
petalbot
dataforseobot
weborama-fetcher
gptbot
| Rule | Path |
|---|---|
| Disallow | / |
mail.ru
msnbot
bingbot
sputnikbot
| Rule | Path |
|---|---|
| Disallow | /company/map/ |
| Disallow | /company/sert/ |
| Disallow | /company/decl/ |
| Disallow | /company/debtor/ |
| Disallow | /company/*/map |
| Disallow | /company/*/sert |
| Disallow | /company/*/decl |
| Disallow | /company/*/debtor |
| Disallow | /company/*/history |
| Disallow | /company/*/report |
| Disallow | /company/*/graph |
| Disallow | /company/*/show/founders |
| Disallow | /company/*/year/ |
| Disallow | /company/*/check |
| Disallow | /company/*/rsmppp |
| Disallow | /company/*/contract |
| Disallow | /company/*/vacancy |
| Disallow | /company/*/trademark |
| Disallow | /company/*/arbitrage |
| Disallow | /company/*/bankruptcy |
| Disallow | /company/*/vestnik |
| Disallow | /company/*/fedresurs |
| Disallow | /man/*/graph |
| Disallow | /man/*/check |
| Disallow | /man/*/rsmppp |
| Disallow | /man/*/contract |
| Disallow | /man/*/trademark |
| Disallow | /man/*/arbitrage |
| Disallow | /man/*/bankruptcy |
| Disallow | /man/*/fedresurs |
| Disallow | /compare/ |
| Disallow | /image/tz/ |
| Disallow | /js/ |
| Disallow | /stat/ |
| Disallow | /decl/ |
| Disallow | /arbitr/ |
| Disallow | /kcap.php |
| Disallow | /ajax.php |
| Disallow | /on_map_ajax.php |
| Disallow | /coord.php |
| Disallow | /go |
| Disallow | /go.php |
| Disallow | /search |
| Disallow | /account |
| Disallow | /request |
| Disallow | /bot |
| Disallow | /terms |
| Disallow | /privacy |
| Disallow | /excel |
| Disallow | /bl_excel.php |
| Disallow | /bl_center.php |
| Disallow | /companies_on_map/ |
| Disallow | /list?*page= |
| Disallow | /mans/*/page/ |
| Disallow | /*adblock* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 2 |
yandex
| Rule | Path |
|---|---|
| Disallow | /company/map/ |
| Disallow | /company/sert/ |
| Disallow | /company/decl/ |
| Disallow | /company/debtor/ |
| Disallow | /company/*/map |
| Disallow | /company/*/sert |
| Disallow | /company/*/decl |
| Disallow | /company/*/debtor |
| Disallow | /company/*/history |
| Disallow | /company/*/report |
| Disallow | /company/*/graph |
| Disallow | /company/*/show/founders |
| Disallow | /company/*/year/ |
| Disallow | /company/*/check |
| Disallow | /company/*/rsmppp |
| Disallow | /company/*/contract*?page= |
| Disallow | /company/*/vacancy |
| Disallow | /company/*/trademark?page= |
| Disallow | /company/*/arbitrage |
| Disallow | /company/*/bankruptcy |
| Disallow | /company/*/vestnik |
| Disallow | /company/*/fedresurs |
| Disallow | /man/*/graph |
| Disallow | /man/*/check |
| Disallow | /man/*/rsmppp |
| Disallow | /man/*/contract*?page= |
| Disallow | /man/*/trademark?page= |
| Disallow | /man/*/arbitrage |
| Disallow | /man/*/bankruptcy |
| Disallow | /man/*/fedresurs |
| Disallow | /compare/ |
| Disallow | /image/tz/ |
| Disallow | /js/ |
| Disallow | /stat/ |
| Disallow | /decl/ |
| Disallow | /arbitr/ |
| Disallow | /kcap.php |
| Disallow | /ajax.php |
| Disallow | /on_map_ajax.php |
| Disallow | /coord.php |
| Disallow | /go |
| Disallow | /go.php |
| Disallow | /search |
| Disallow | /account |
| Disallow | /request |
| Disallow | /bot |
| Disallow | /terms |
| Disallow | /privacy |
| Disallow | /excel |
| Disallow | /bl_excel.php |
| Disallow | /bl_center.php |
| Disallow | /companies_on_map/ |
| Disallow | /list?*page= |
| Disallow | /mans/*/page/ |
| Disallow | /*adblock* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 0.2 |
*
| Rule | Path |
|---|---|
| Disallow | /company/map/ |
| Disallow | /company/sert/ |
| Disallow | /company/decl/ |
| Disallow | /company/debtor/ |
| Disallow | /company/*/map |
| Disallow | /company/*/sert |
| Disallow | /company/*/decl |
| Disallow | /company/*/debtor |
| Disallow | /company/*/history |
| Disallow | /company/*/report |
| Disallow | /company/*/graph |
| Disallow | /company/*/show/founders |
| Disallow | /company/*/year/ |
| Disallow | /company/*/check |
| Disallow | /company/*/rsmppp |
| Disallow | /company/*/contract*?page= |
| Disallow | /company/*/vacancy |
| Disallow | /company/*/trademark?page= |
| Disallow | /company/*/arbitrage |
| Disallow | /company/*/bankruptcy |
| Disallow | /company/*/vestnik |
| Disallow | /company/*/fedresurs?page= |
| Disallow | /man/*/graph |
| Disallow | /man/*/check |
| Disallow | /man/*/rsmppp |
| Disallow | /man/*/contract*?page= |
| Disallow | /man/*/trademark?page= |
| Disallow | /man/*/arbitrage |
| Disallow | /man/*/bankruptcy |
| Disallow | /man/*/fedresurs?page= |
| Disallow | /image/tz/ |
| Disallow | /stat/ |
| Disallow | /decl/ |
| Disallow | /kcap.php |
| Disallow | /ajax.php |
| Disallow | /on_map_ajax.php |
| Disallow | /coord.php |
| Disallow | /go |
| Disallow | /go.php |
| Disallow | /search |
| Disallow | /account |
| Disallow | /request |
| Disallow | /bot |
| Disallow | /terms |
| Disallow | /privacy |
| Disallow | /excel |
| Disallow | /bl_center.php |
| Disallow | /companies_on_map/ |
| Disallow | /list?*page= |
| Disallow | /mans/*/page/ |
| Disallow | /*adblock* |
Warnings
- `clean-param` is not a known field.
- `host` is not a known field.