worstoftheweb.com
robots.txt
Robots Exclusion Standard data for worstoftheweb.com
Resource Scan
Scan Details
| Site Domain | worstoftheweb.com |
| Base Domain | worstoftheweb.com |
| Scan Status | Ok |
| Last Scan | 2025-12-20T06:48:38+00:00 |
| Next Scan | 2026-01-19T06:48:38+00:00 |
Last Scan
| Scanned | 2025-12-20T06:48:38+00:00 |
| URL | https://worstoftheweb.com/robots.txt |
| Redirect | http://www.worstoftheweb.com/robots.txt |
| Redirect Domain | www.worstoftheweb.com |
| Redirect Base | worstoftheweb.com |
| Domain IPs | 104.21.61.45, 172.67.206.3, 2606:4700:3030::ac43:ce03, 2606:4700:3036::6815:3d2d |
| Redirect IPs | 104.21.61.45, 172.67.206.3, 2606:4700:3030::ac43:ce03, 2606:4700:3036::6815:3d2d |
| Response IP | 104.21.61.45 |
| Found | Yes |
| Hash | 59cefedb2ab1bd9f7ea2af13935a7e765263cb31f05559073b842d17a436efa0 |
| SimHash | 0d7500495d01 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /cgi-bin/ |
| Disallow | /adbanners/ |
| Disallow | /adclicks/ |
| Disallow | /adstats/ |
| Disallow | /newstats/ |
| Disallow | /Graphics/ |
| Disallow | /calendar.txt |
| Disallow | /suggestions/ |
Comments