di01.ru
robots.txt
Robots Exclusion Standard data for di01.ru
Resource Scan
Scan Details
| Site Domain | di01.ru |
| Base Domain | di01.ru |
| Scan Status | Ok |
| Last Scan | 2024-08-25T11:00:35+00:00 |
| Next Scan | 2024-09-24T11:00:35+00:00 |
Last Scan
| Scanned | 2024-08-25T11:00:35+00:00 |
| URL | https://di01.ru/robots.txt |
| Domain IPs | 2a00:15f8:a000:5:1:11:8:7bd, 2a00:15f8:a000:5:1:12:8:7bd, 2a00:15f8:a000:5:1:13:8:7bd, 2a00:15f8:a000:5:1:14:8:7bd, 90.156.201.108, 90.156.201.14, 90.156.201.20, 90.156.201.51 |
| Response IP | 90.156.201.51 |
| Found | Yes |
| Hash | 317982bb795dd811df57345835dbac8e43757e3b4bbb98c30a67e5ca0776710e |
| SimHash | 4b6cd673c638 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /my/ |
| Disallow | /checkout/ |
| Disallow | /my/ |
| Disallow | /checkout/ |
| Disallow | /search/?query= |
| Disallow | /compare/ |
| Disallow | /tag/ |
| Disallow | *%26sort%3D |
| Disallow | */?sort= |
| Disallow | */?etext= |
| Disallow | /cart/ |
| Disallow | /checkout/ |
| Disallow | /my/ |
| Disallow | /signup/ |
| Disallow | /login/ |
| Disallow | /order/ |
| Disallow | /ok/ |
| Disallow | /forgotpassword/ |
| Disallow | /webasyst/ |
| Disallow | /parser/ |
| Disallow | /mailer/unsubscribe/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 120 |
yandex
| Rule | Path |
|---|---|
| Disallow | /search/?query= |
| Disallow | /compare/ |
| Disallow | /tag/ |
| Disallow | *%26sort%3D |
| Disallow | */?sort= |
| Disallow | */?etext= |
| Disallow | /cart/ |
| Disallow | /checkout/ |
| Disallow | /my/ |
| Disallow | /signup/ |
| Disallow | /login/ |
| Disallow | /order/ |
| Disallow | /ok/ |
| Disallow | /forgotpassword/ |
| Disallow | /webasyst/ |
| Disallow |
Other Records
| Field | Value |
|---|---|
| sitemap | https://di01.ru/sitemap.xml |
| sitemap | https://di01.ru/sitemap.xml |
| sitemap | https://di01.ru/sitemap/ |
Warnings
- `clean-param` is not a known field.
- `host` is not a known field.
Comments