stihi-devushke.ru
robots.txt
Robots Exclusion Standard data for stihi-devushke.ru
Resource Scan
Scan Details
| Site Domain | stihi-devushke.ru |
| Base Domain | stihi-devushke.ru |
| Scan Status | Ok |
| Last Scan | 2024-10-07T07:25:58+00:00 |
| Next Scan | 2024-10-14T07:25:58+00:00 |
Last Scan
| Scanned | 2024-10-07T07:25:58+00:00 |
| URL | https://stihi-devushke.ru/robots.txt |
| Domain IPs | 87.236.16.27 |
| Response IP | 87.236.16.27 |
| Found | Yes |
| Hash | 19eb219316dc33619073860f62014e591f792fbbf0710e05e732fe7276850fa3 |
| SimHash | 6b0c9e42ce21 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /auth/* |
| Disallow | /shelf |
| Disallow | /shelf/* |
| Disallow | /profile |
| Disallow | /profile/* |
| Disallow | /external_book |
| Disallow | /external_book/* |
| Disallow | /messages |
| Disallow | /messages/* |
| Disallow | /scores* |
| Disallow | /comments* |
| Disallow | /likes* |
| Disallow | /views* |
| Disallow | /blog |
| Disallow | /blog/* |
| Disallow | /shop |
| Disallow | /shop/* |
| Disallow | /print/* |
| Disallow | /*?* |
| Allow | / |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 1 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://istihi.ru/sitemap.xml |
Warnings
- `host` is not a known field.
- `request-rate` is not a known field.