boxlunch.com
robots.txt
Robots Exclusion Standard data for boxlunch.com
Resource Scan
Scan Details
| Site Domain | boxlunch.com |
| Base Domain | boxlunch.com |
| Scan Status | Ok |
| Last Scan | 2024-10-05T21:46:13+00:00 |
| Next Scan | 2024-10-19T21:46:13+00:00 |
Last Scan
| Scanned | 2024-10-05T21:46:13+00:00 |
| URL | https://boxlunch.com/robots.txt |
| Redirect | https://www.boxlunch.com/robots.txt |
| Redirect Domain | www.boxlunch.com |
| Redirect Base | boxlunch.com |
| Domain IPs | 104.17.136.184, 104.17.137.184 |
| Redirect IPs | 104.17.136.184, 104.17.137.184 |
| Response IP | 104.17.136.184 |
| Found | Yes |
| Hash | 2e973564c0bdd5d27f5e8bc2e7edcce303f7f9d48191a4977065baf2a0093247 |
| SimHash | 233457775e16 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /OOBO/ |
| Disallow | /mkt/ |
| Disallow | /*pmax%3D* |
| Disallow | /*pmin%3D* |
| Disallow | /*prefn* |
| Disallow | /*prefv* |
| Disallow | /*%26distance* |
| Disallow | /*pmid%3D* |
| Disallow | /*srule%3D* |
| Disallow | /*queryBRT%3D* |
| Disallow | /*?q=* |
| Allow | /*theme?q=* |
| Allow | /tees-2/?q=* |
| Disallow | /c/* |
| Allow | /pop-culture/*/c/* |
| Disallow | /pop-culture/shop-by-license/c/* |
| Allow | /pop-culture/shop-by-license/*/c/* |
| Disallow | */in-store-deals* |
| Disallow | /*dwcont%3D* |
| Disallow | /*pid%3D* |
| Disallow | /*Stores-STS* |
| Disallow | /*Product-ShowQuickView* |
| Disallow | /*Product-Variation* |
| Disallow | /*Wishlist-Add* |
| Disallow | /*_*/ |
| Allow | /product/*_*.html |
| Disallow | /account* |
| Disallow | /cart* |
| Disallow | /checkout* |
| Disallow | /order-history* |
| Disallow | /search* |
| Disallow | /wishlist* |
| Disallow | */demandware.store* |
| Allow | */demandware.store/*/Search-UpdateGrid |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.boxlunch.com/sitemap_index.xml |
Comments