welib.st
robots.txt
Robots Exclusion Standard data for welib.st
Resource Scan
Scan Details
| Site Domain | welib.st |
| Base Domain | welib.st |
| Scan Status | Ok |
| Last Scan | 2025-11-22T17:55:14+00:00 |
| Next Scan | 2025-11-29T17:55:14+00:00 |
Last Scan
| Scanned | 2025-11-22T17:55:14+00:00 |
| URL | https://welib.st/robots.txt |
| Domain IPs | 104.21.5.239, 172.67.154.196, 2606:4700:3031::6815:5ef, 2606:4700:3031::ac43:9ac4 |
| Response IP | 104.21.5.239 |
| Found | Yes |
| Hash | 1dece8642cf48e2707c187fb5354885721c23ffa47c44bbbd67f29eaff8878db |
| SimHash | 6291f6a845c8 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /db |
| Disallow | /account |
| Disallow | /contact |
| Disallow | /login |
| Disallow | /donate |
| Disallow | /copyright |
| Disallow | /popular |
| Disallow | /forgot-password |
| Disallow | /md5 |
| Disallow | /slow_download |
| Disallow | /fast_preview |
| Disallow | /slow_download |
| Disallow | /list/ |
| Disallow | /fast_download |
| Disallow | /torrents |
| Disallow | /search |
| Disallow | /scidb |
| Disallow | /member_codes |
| Disallow | /codes |
| Disallow | /dyn/ |
| Disallow | /ipfs_downloads/ |
| Disallow | /fast_preview/ |
| Disallow | /ia/ |
| Disallow | /faq |
| Disallow | /datasets |
| Disallow | /browser_verification |
| Disallow | /blog/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |