thelib.net
robots.txt
Robots Exclusion Standard data for thelib.net
Resource Scan
Scan Details
| Site Domain | thelib.net |
| Base Domain | thelib.net |
| Scan Status | Ok |
| Last Scan | 2025-10-06T00:34:17+00:00 |
| Next Scan | 2025-11-05T00:34:17+00:00 |
Last Scan
| Scanned | 2025-10-06T00:34:17+00:00 |
| URL | https://thelib.net/robots.txt |
| Domain IPs | 104.21.86.76, 172.67.216.147, 2606:4700:3030::ac43:d893, 2606:4700:3035::6815:564c |
| Response IP | 104.21.86.76 |
| Found | Yes |
| Hash | c2d2ce6aeb936fcad811d54974a86996f08bef372c85f4d2b429ae7e5da9f084 |
| SimHash | fd0db0788433 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /engine/go.php |
| Disallow | /user/ |
| Disallow | /newposts/ |
| Disallow | /statistics.html |
| Disallow | /*subaction%3Duserinfo |
| Disallow | /*subaction%3Dnewposts |
| Disallow | /*do%3Dlastcomments |
| Disallow | /*do%3Dfeedback |
| Disallow | /*do%3Dregister |
| Disallow | /*do%3Dlostpassword |
| Disallow | /*do%3Daddnews |
| Disallow | /*do%3Dstats |
| Disallow | /*do%3Dpm |
| Disallow | /*do%3Dsearch |
| Disallow | /*do%3Ddownload |
| Disallow | /*do%3Dgo |
Other Records
| Field | Value |
|---|---|
| sitemap | https://thelib.net/sitemap.xml |