shikimori.one
robots.txt
Robots Exclusion Standard data for shikimori.one
Resource Scan
Scan Details
| Site Domain | shikimori.one |
| Base Domain | shikimori.one |
| Scan Status | Ok |
| Last Scan | 2024-10-02T22:15:12+00:00 |
| Next Scan | 2024-10-09T22:15:12+00:00 |
Last Scan
| Scanned | 2024-10-02T22:15:12+00:00 |
| URL | https://shikimori.one/robots.txt |
| Domain IPs | 104.21.81.61, 172.67.157.46, 2606:4700:3030::6815:513d, 2606:4700:3036::ac43:9d2e |
| Response IP | 104.21.81.61 |
| Found | Yes |
| Hash | b06ad7a99cc790b6ebc9963842ff9be3b6d91f9d83ecb5b0a148c260dc3ee2f6 |
| SimHash | c2148ab6c6a1 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /*? |
| Disallow | /*?rel=nofollow |
| Disallow | /cosplay/* |
| Disallow | /genres |
| Disallow | /animes/*rss |
| Disallow | /animes/order-by/* |
| Disallow | /mangas/*rss |
| Disallow | /mangas/order-by/* |
| Disallow | /animes/search/* |
| Disallow | /mangas/search/* |
| Disallow | /*/comments |
| Disallow | /*/tooltip |
| Disallow | /*/autocomplete |
| Disallow | /*/autocomplete/v2 |
| Disallow | /groups/9-Hentai* |
| Disallow | /messages/* |
| Disallow | /*undefined |
| Disallow | /api/* |
| Disallow | /*.html |
| Disallow | /*/*.rss |
| Disallow | /clubs/3* |
Other Records
| Field | Value |
|---|---|
| sitemap | https://shikimori.one/sitemap.xml |
Warnings
- `host` is not a known field.