cosmicthings.ru
robots.txt
Robots Exclusion Standard data for cosmicthings.ru
Resource Scan
Scan Details
| Site Domain | cosmicthings.ru |
| Base Domain | cosmicthings.ru |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Couldn't connect to server. |
| Last Scan | 2024-08-31T20:13:26+00:00 |
| Next Scan | 2024-11-29T20:13:26+00:00 |
Last Successful Scan
| Scanned | 2023-10-14T18:20:49+00:00 |
| URL | https://cosmicthings.ru/robots.txt |
| Domain IPs | 104.21.89.192, 172.67.191.10, 2606:4700:3035::6815:59c0, 2606:4700:3035::ac43:bf0a |
| Response IP | 104.21.89.192 |
| Found | Yes |
| Hash | c69e9ccbdba675ff11a97415e02990a44a2e7c070b9623a31bb6fa39db968283 |
| SimHash | 2e79d8c561f2 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /songs/page/* |
| Disallow | /let/* |
| Disallow | /genre/* |
| Disallow | /download/* |
| Disallow | /s/p/d |
| Disallow | /s/p/w |
| Disallow | /s/p/m |
| Disallow | /s/p/a |
| Disallow | /s/mr |
| Disallow | /s/mc |
| Disallow | /playlists |
| Disallow | /playlist/* |
| Disallow | /muzs/page/* |
| Disallow | /bkv/* |
| Disallow | /listen/* |
| Disallow | /films/tag/* |
| Disallow | /video/d/* |
| Disallow | /films/p/d |
| Disallow | /films/p/w |
| Disallow | /films/p/m |
| Disallow | /films/p/a |
| Disallow | /films/mr |
| Disallow | /films/mc |
| Disallow | /photos/tag/* |
| Disallow | /photos/d/* |
| Disallow | /photos/p/d |
| Disallow | /photos/p/w |
| Disallow | /photos/p/m |
| Disallow | /photos/p/a |
| Disallow | /photos/mr |
| Disallow | /photos/mc |
| Disallow | /torrents/tag/* |
| Disallow | /torrents/d/* |
| Disallow | /torrents/p/d |
| Disallow | /torrents/p/w |
| Disallow | /torrents/p/m |
| Disallow | /torrents/p/a |
| Disallow | /torrents/mr |
| Disallow | /torrents/mc |
| Disallow | /login |
| Disallow | /reg |
| Disallow | /forgotpass |
| Disallow | /search |
| Disallow | /feedback |
| Disallow | /guestbook |
| Disallow | /uploads/captcha/* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
Other Records
| Field | Value |
|---|---|
| sitemap | http://cosmicthings.ru/sitemap.xml |
Warnings
- `host` is not a known field.