thesaurus.com
robots.txt
Robots Exclusion Standard data for thesaurus.com
Resource Scan
Scan Details
| Site Domain | thesaurus.com |
| Base Domain | thesaurus.com |
| Scan Status | Ok |
| Last Scan | 2024-10-30T14:24:40+00:00 |
| Next Scan | 2024-11-06T14:24:40+00:00 |
Last Scan
| Scanned | 2024-10-30T14:24:40+00:00 |
| URL | https://thesaurus.com/robots.txt |
| Redirect | https://www.thesaurus.com/robots.txt |
| Redirect Domain | www.thesaurus.com |
| Redirect Base | thesaurus.com |
| Domain IPs | 151.101.130.133, 151.101.194.133, 151.101.2.133, 151.101.66.133, 2a04:4e42:200::645, 2a04:4e42:400::645, 2a04:4e42:600::645, 2a04:4e42::645 |
| Redirect IPs | 151.101.130.133, 151.101.194.133, 151.101.2.133, 151.101.66.133, 2a04:4e42:200::645, 2a04:4e42:400::645, 2a04:4e42:600::645, 2a04:4e42::645 |
| Response IP | 199.232.46.133 |
| Found | Yes |
| Hash | 9c863059f89ad61c5ed799dd1b1687b48183ab06d2c9072262533a445d036dc8 |
| SimHash | 2a0798a0cb32 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /5480.iac. |
| Disallow | /go/ |
| Disallow | /audio.html/ |
| Disallow | /houseads/ |
| Disallow | /askhome/ |
| Disallow | /t2opt/ |
| Disallow | /browse-2/ |
| Allow | / |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.thesaurus.com/sitemap-thes-index.xml |
| sitemap | https://www.thesaurus.com/e/sitemap_index.xml |
Comments