sweepico.com
robots.txt
Robots Exclusion Standard data for sweepico.com
Resource Scan
Scan Details
| Site Domain | sweepico.com |
| Base Domain | sweepico.com |
| Scan Status | Ok |
| Last Scan | 2026-01-25T23:23:11+00:00 |
| Next Scan | 2026-02-24T23:23:11+00:00 |
Last Scan
| Scanned | 2026-01-25T23:23:11+00:00 |
| URL | https://sweepico.com/robots.txt |
| Domain IPs | 104.18.32.154, 172.64.155.102, 2606:4700:4407::ac40:9b66, 2a06:98c1:3101::6812:209a |
| Response IP | 172.64.155.102 |
| Found | Yes |
| Hash | 42bfa3ae67ce26f89ca375c95ded68bc9bbf86c2149ba110631160fe85af891e |
| SimHash | 2a00794ac738 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /*?* |
| Disallow | /deep-link/* |
| Disallow | /lobby?* |
| Disallow | /auth/* |
| Disallow | /signup/* |
| Disallow | /signin/* |
| Disallow | /feed/ |
| Disallow | *action%3D |
| Disallow | /news/?p |
| Disallow | /articles/?p |
| Disallow | /affiliates |
| Disallow | /registration-complete |
| Disallow | /__prometheus_metrics |
| Disallow | /__prometheus_health |