cnews.ru
robots.txt
Robots Exclusion Standard data for cnews.ru
Resource Scan
Scan Details
| Site Domain | cnews.ru |
| Base Domain | cnews.ru |
| Scan Status | Ok |
| Last Scan | 2024-11-12T17:34:46+00:00 |
| Next Scan | 2024-11-19T17:34:46+00:00 |
Last Scan
| Scanned | 2024-11-12T17:34:46+00:00 |
| URL | https://cnews.ru/robots.txt |
| Redirect | https://www.cnews.ru/robots.txt |
| Redirect Domain | www.cnews.ru |
| Redirect Base | cnews.ru |
| Domain IPs | 89.108.90.34 |
| Redirect IPs | 89.108.90.34 |
| Response IP | 89.108.90.34 |
| Found | Yes |
| Hash | d35474b7c621f256c6d19e9b7a396ad398caa76c4f89a91c79b8806107a7bcd6 |
| SimHash | 7d14a5456e90 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /main/ |
| Disallow | /news/for_blog/ |
| Disallow | /articles/for_blog/ |
| Disallow | /news/for_print/ |
| Disallow | /articles/for_print/ |
| Disallow | /articles/preview/ |
| Disallow | /news/preview/ |
| Disallow | /reviews/preview_articles/ |
| Disallow | /reviews/preview_articles/ |
| Disallow | /redirect.php |
| Disallow | /search |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.cnews.ru/inc/sitemap.xml |
| sitemap | https://www.cnews.ru/inc/sitemap_book.xml |
Warnings
- `host` is not a known field.