wineonline.ca
robots.txt
Robots Exclusion Standard data for wineonline.ca
Resource Scan
Scan Details
| Site Domain | wineonline.ca |
| Base Domain | wineonline.ca |
| Scan Status | Ok |
| Last Scan | 2024-11-06T05:26:34+00:00 |
| Next Scan | 2024-11-20T05:26:34+00:00 |
Last Scan
| Scanned | 2024-11-06T05:26:34+00:00 |
| URL | https://wineonline.ca/robots.txt |
| Redirect | https://www.wineonline.ca/robots.txt |
| Redirect Domain | www.wineonline.ca |
| Redirect Base | wineonline.ca |
| Domain IPs | 162.159.136.54, 162.159.137.54 |
| Redirect IPs | 162.159.136.54, 162.159.137.54 |
| Response IP | 162.159.136.54 |
| Found | Yes |
| Hash | ffc96c1d6809016c958b5e03a9f0ad2aed4c087dd64b4f8d4e9a4adcca036bad |
| SimHash | 2c507c65d591 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /lander/* |
| Disallow | /kaneff |
| Disallow | /visa |
| Disallow | /amex.html |
| Disallow | /25-4oz-meier-s-pink-catawba/* |
| Disallow | /aereo-wine-aerator-by-true/* |
| Disallow | /sambadays/* |
| Disallow | /exitintent/index/* |
| Disallow | /offers-archive/* |
| Disallow | /firecheckout/ |
| Disallow | /certified-checkout/ |
| Disallow | /penguin/* |
| Disallow | /white-wine-for-dummies-by-wiley/* |
| Disallow | /js/* |
| Disallow | /skin/* |
| Disallow | /dealerinquiry/* |
| Disallow | /wishlist/* |
| Disallow | /*?cluid=* |
| Disallow | /*%26cluid%3D* |
| Disallow | /*search?q=* |
| Disallow | /*search/?q=* |
| Disallow | /*catalogsearch/result?q=* |
| Disallow | /*catalogsearch/result/?q=* |
| Disallow | /.well-known/* |
| Disallow | /winefest2024 |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 30 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.wineonline.ca/sitemap_on.xml |
| sitemap | https://www.wineonline.ca/disabled_sitemap_ontario.xml |