compthesaurus.com
robots.txt
Robots Exclusion Standard data for compthesaurus.com
Resource Scan
Scan Details
| Site Domain | compthesaurus.com |
| Base Domain | compthesaurus.com |
| Scan Status | Ok |
| Last Scan | 6/8/2025, 4:33:37 AM |
| Next Scan | 6/15/2025, 4:33:37 AM |
Last Scan
| Scanned | 6/8/2025, 4:33:37 AM |
| URL | https://compthesaurus.com/robots.txt |
| Domain IPs | 104.21.65.3, 172.67.138.173, 2606:4700:3034::6815:4103, 2606:4700:3035::ac43:8aad |
| Response IP | 172.67.138.173 |
| Found | Yes |
| Hash | 3a03f25565cec160136cf66f7e3a9fed692dc3c72a813026722e5e392fb85633 |
| SimHash | 6940ba40cfd0 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | *utm%3D |
| Disallow | *clid%3D |
| Disallow | *openstat%3D |
| Disallow | *from%3D |
| Disallow |
Other Records
| Field | Value |
|---|---|
| sitemap | https://compthesaurus.com/sitemap.xml |
Warnings
- `host` is not a known field.