tarantulaforum.com
robots.txt
Robots Exclusion Standard data for tarantulaforum.com
Resource Scan
Scan Details
| Site Domain | tarantulaforum.com |
| Base Domain | tarantulaforum.com |
| Scan Status | Ok |
| Last Scan | 2025-08-02T06:53:10+00:00 |
| Next Scan | 2025-08-09T06:53:10+00:00 |
Last Scan
| Scanned | 2025-08-02T06:53:10+00:00 |
| URL | https://tarantulaforum.com/robots.txt |
| Domain IPs | 104.21.53.57, 172.67.209.38, 2606:4700:3033::ac43:d126, 2606:4700:3037::6815:3539 |
| Response IP | 104.21.53.57 |
| Found | Yes |
| Hash | dcaa2ba23cff0ade4c01846558022d7fe561f92286233455518df9596e84c646 |
| SimHash | 5052cc70e3d3 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /find-new/ |
| Disallow | /account/ |
| Disallow | /attachments/ |
| Disallow | /goto/ |
| Disallow | /posts/ |
| Disallow | /login/ |
| Disallow | /admin.php |
| Allow | / |
| Disallow | */unread$ |
| Disallow | */latest$ |
| Allow | /search/$ |
| Disallow | /search/* |
| Allow | */threads/post-* |
| Disallow | */post-* |