crockpot.com
robots.txt
Robots Exclusion Standard data for crockpot.com
Resource Scan
Scan Details
| Site Domain | crockpot.com |
| Base Domain | crockpot.com |
| Scan Status | Ok |
| Last Scan | 2024-08-23T03:40:41+00:00 |
| Next Scan | 2024-09-22T03:40:41+00:00 |
Last Scan
| Scanned | 2024-08-23T03:40:41+00:00 |
| URL | https://crockpot.com/robots.txt |
| Redirect | https://www.crock-pot.com/robots.txt |
| Redirect Domain | www.crock-pot.com |
| Redirect Base | crock-pot.com |
| Domain IPs | 104.18.39.235, 172.64.148.21, 2606:4700:4400::6812:27eb, 2606:4700:4400::ac40:9415 |
| Redirect IPs | 104.18.41.224, 172.64.146.32, 2606:4700:4400::6812:29e0, 2606:4700:4400::ac40:9220 |
| Response IP | 172.64.146.32 |
| Found | Yes |
| Hash | d4ed67b6649e227b98152462ac0746e23692e3074c01651f6168986171eda71c |
| SimHash | 61d568424f55 |
Groups
*
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /*cart |
| Disallow | /account/ |
| Disallow | /setpassword/ |
| Disallow | /search* |
| Disallow | /confirmednewpassword/ |
| Disallow | /profile/ |
| Disallow | /orders/ |
| Disallow | *demandware.store* |
| Disallow | *pmax* |
| Disallow | *pmin* |
| Disallow | /*prefn*%3D |
| Disallow | /*prefv*%3D |
| Disallow | *q%3D* |
| Disallow | *srule%3D* |
| Disallow | *format%3Dajax* |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.crock-pot.com/sitemap_index.xml |