thayers.com
robots.txt
Robots Exclusion Standard data for thayers.com
Resource Scan
Scan Details
| Site Domain | thayers.com |
| Base Domain | thayers.com |
| Scan Status | Ok |
| Last Scan | 2024-09-27T16:30:01+00:00 |
| Next Scan | 2024-10-11T16:30:01+00:00 |
Last Scan
| Scanned | 2024-09-27T16:30:01+00:00 |
| URL | https://thayers.com/robots.txt |
| Redirect | https://www.thayers.com/robots.txt |
| Redirect Domain | www.thayers.com |
| Redirect Base | thayers.com |
| Domain IPs | 104.18.34.190, 172.64.153.66, 2606:4700:4400::6812:22be, 2606:4700:4400::ac40:9942 |
| Redirect IPs | 104.18.34.190, 172.64.153.66, 2606:4700:4400::6812:22be, 2606:4700:4400::ac40:9942 |
| Response IP | 104.18.34.190 |
| Found | Yes |
| Hash | 0c28cb3fdf45781b5a2ce9924f6a94e798ec7c7dcbcda7a073372b6fbd4edbd1 |
| SimHash | f6be1424e1d7 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /cart* |
| Disallow | /account* |
| Disallow | /register |
| Disallow | /password* |
| Disallow | /addresses* |
| Disallow | /billing* |
| Disallow | /order* |
| Disallow | /my-order-history |
| Disallow | /wishlist* |
| Disallow | /favorites |
| Disallow | /search* |
| Disallow | /logout |
| Disallow | /checkout-start* |
| Disallow | /*dwcont |
| Disallow | /*format |
| Disallow | /*linksrc |
| Disallow | /*pmin |
| Disallow | /*pmax |
| Disallow | /*prefn |
| Disallow | /*prefv |
| Disallow | /*lng |
| Disallow | /*lat |
| Disallow | /*srule |
| Disallow | /*subid |
| Disallow | /*sz |
| Disallow | /*variantID |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 5 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.thayers.com/sitemap_index.xml |