ssl.dlsite.com
robots.txt
Robots Exclusion Standard data for ssl.dlsite.com
Resource Scan
Scan Details
| Site Domain | ssl.dlsite.com |
| Base Domain | dlsite.com |
| Scan Status | Ok |
| Last Scan | 2024-04-22T04:58:14+00:00 |
| Next Scan | 2024-05-22T04:58:14+00:00 |
Last Scan
| Scanned | 2024-04-22T04:58:14+00:00 |
| URL | https://ssl.dlsite.com/robots.txt |
| Domain IPs | 18.155.68.29, 18.155.68.83, 18.155.68.87, 18.155.68.91 |
| Response IP | 18.155.68.87 |
| Found | Yes |
| Hash | 035aac72cf71864cbea242d9bf029f63f196dcac36fa3aa605d9f5ad55fc28ff |
| SimHash | 6f3486604b22 |
Groups
bingbot
| Rule | Path |
|---|---|
| Disallow | */fsr/%3D/*/per_page/*/page/* |
| Allow | / |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 5 |
*
| Rule | Path |
|---|---|
| Disallow | /*module |
| Disallow | /*action |
| Disallow | /*adultcheck |
| Disallow | /error2.html |
| Disallow | /*mypage |
| Disallow | /*simura |
| Disallow | /*search/result |
| Disallow | /*maker/profile/dlblog/content |
| Disallow | /hana/* |
| Disallow | /booksl/* |
| Disallow | /pro2/* |
| Disallow | */dealer/* |
| Disallow | */load/recommend/* |
| Disallow | */load/review/* |
| Disallow | */article/list/* |
| Disallow | */work/image/sample/* |
| Disallow | /js/adultcheck.js |
| Disallow | */load/favorite/product |
| Disallow | */load/not/interest |
| Disallow | */load/bought/product |
| Disallow | */user/product/smmary/%3D/products/ |
| Disallow | */announce/summary |
| Disallow | /circle/profile/ |
| Disallow | */author/list/ |
| Disallow | /eisys/blog/auth.php |
| Disallow | /work/%3D/product_id/ |
| Disallow | /work/reviewlist/ |
| Disallow | /popup/ |
| Allow | / |
| Allow | */mypage/wishlist |
| Allow | */mypage/followlist |
| Allow | */load/recommend/%3D/* |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.dlsite.com/maniax/sitemap/index/xml |
| sitemap | https://www.dlsite.com/home/sitemap/index/xml |
| sitemap | https://www.dlsite.com/books/sitemap/index/xml |
| sitemap | https://www.dlsite.com/comic/sitemap/index/xml |
| sitemap | https://www.dlsite.com/pro/sitemap/index/xml |
| sitemap | https://www.dlsite.com/soft/sitemap/index/xml |
| sitemap | https://www.dlsite.com/girls/sitemap/index/xml |
| sitemap | https://www.dlsite.com/girls-pro/sitemap/index/xml |
| sitemap | https://www.dlsite.com/girls-drama/sitemap/index/xml |
| sitemap | https://www.dlsite.com/bl/sitemap/index/xml |
| sitemap | https://www.dlsite.com/bl-pro/sitemap/index/xml |
| sitemap | https://www.dlsite.com/bl-drama/sitemap/index/xml |
Comments