malish-butik.ru
robots.txt
Robots Exclusion Standard data for malish-butik.ru
Resource Scan
Scan Details
| Site Domain | malish-butik.ru |
| Base Domain | malish-butik.ru |
| Scan Status | Ok |
| Last Scan | 2024-10-31T05:23:37+00:00 |
| Next Scan | 2024-11-30T05:23:37+00:00 |
Last Scan
| Scanned | 2024-10-31T05:23:37+00:00 |
| URL | http://malish-butik.ru/robots.txt |
| Domain IPs | 2a00:15f8:a000:5:1:11:8:3cfa, 2a00:15f8:a000:5:1:12:8:3cfa, 2a00:15f8:a000:5:1:13:8:3cfa, 2a00:15f8:a000:5:1:14:8:3cfa, 90.156.201.103, 90.156.201.39, 90.156.201.42, 90.156.201.83 |
| Response IP | 90.156.201.39 |
| Found | Yes |
| Hash | aff095670cbdfd2f7ca70b8a685e97d12077306c1cfaf47759c045d8609cc4b0 |
| SimHash | e71898628a32 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /my/ |
| Disallow | /checkout/ |
| Disallow | /cart/ |
| Disallow | /search/ |
| Disallow | /tag/* |
| Disallow | /mailer/unsubscribe/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 1 |
Other Records
| Field | Value |
|---|---|
| sitemap | http://malish-butik.ru/sitemap.xml |
Warnings
- `host` is not a known field.
Comments