text-lyrics.ru
robots.txt
Robots Exclusion Standard data for text-lyrics.ru
Resource Scan
Scan Details
| Site Domain | text-lyrics.ru |
| Base Domain | text-lyrics.ru |
| Scan Status | Ok |
| Last Scan | 2024-06-28T08:55:33+00:00 |
| Next Scan | 2024-07-05T08:55:33+00:00 |
Last Scan
| Scanned | 2024-06-28T08:55:33+00:00 |
| URL | https://text-lyrics.ru/robots.txt |
| Domain IPs | 95.142.35.91 |
| Response IP | 95.142.35.91 |
| Found | Yes |
| Hash | 3dc61841569170fe16f7e51674810be34f07dce7ebc825f73cc00fa4c9ddfb02 |
| SimHash | 750cfd501431 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /engine/go.php |
| Disallow | /engine/download.php |
| Disallow | /tags/ |
| Disallow | /newposts/ |
| Disallow | /statistics.html |
| Disallow | /*subaction%3Duserinfo |
| Disallow | /*subaction%3Dnewposts |
| Disallow | /*do%3Dlastcomments |
| Disallow | /*do%3Dfeedback |
| Disallow | /*do%3Dregister |
| Disallow | /*do%3Dlostpassword |
| Disallow | /*do%3Daddnews |
| Disallow | /*do%3Dstats |
| Disallow | /*do%3Dpm |
| Disallow | /*do%3Dsearch |
| Disallow | /*do%3Dtags |
| Disallow | /download-pesni-mp3/ |
| Disallow | /*/clips/ |
| Disallow | /*/your-poem/ |
| Disallow | /your-poem/ |
| Disallow | /translate/ |
| Disallow | /k1/krovostok/22 |
| Disallow | /k/krestall/courier/ |
| Disallow | /b/bakaro-/ |
| Disallow | /c1/sever.17/ |
| Disallow | /11854-empty-article.html |
| Disallow | /c1/slava-kpss/ |
| Disallow | /i1/imploziya/clips/ |
| Disallow | /download-pesni-mp3/739-kraym-volshebnik-da-gudda-jazz-esche-one-day-download-.html |
| Disallow | /k/kavabanga/page/3/your-poem/ |
| Disallow | /k/kavabanga/page/3/clips/ |
| Disallow | /top-music/ |
| Disallow | /temp/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
Warnings
- 2 invalid lines.
- `host` is not a known field.