Log In Sign Up

waitrose.com
robots.txt

Robots Exclusion Standard data for waitrose.com

Resource Scan

Scan Details

Site Domain waitrose.com
Base Domain waitrose.com
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2025-11-23T00:06:11+00:00
Next Scan 2026-01-22T00:06:11+00:00

Last Successful Scan

Scanned2025-08-28T19:39:16+00:00
URL https://waitrose.com/robots.txt
Redirect https://www.waitrose.com/robots.txt
Redirect Domain www.waitrose.com
Redirect Base waitrose.com
Domain IPs 23.50.88.178
Redirect IPs 23.50.88.178
Response IP 23.199.143.13
Found Yes
Hash 13c71439b45d930f5d7c2a434321a31608a459e44b148f9d7a93a3b10e7c0cb4
SimHash 6d363d5f1ab4

Groups

*

Rule Path
Disallow /shop/ViewTrolley
Disallow /shop/RegistrationInitiate
Disallow /shop/CheckOutInitiateCmd
Disallow /shop/CheckOutOrderPreviewCmd
Disallow /shop/CheckOutPaymentCmd
Disallow /shop/OrderProcess
Disallow /shop/InitiateLogin
Disallow /shop/RegistrationInitiate
Disallow /shop/InitiateMyAccount
Disallow /shop/LoginDetailsInitiateCmd
Disallow /shop/PrivateInfoInitiateCmd
Disallow /shop/AddressInitiateCmd
Disallow /shop/GetPaymentBillingDetailsCmd
Disallow /shop/OrderStatusDisplayCmd
Disallow /shop/InterestItemDisplayCmd
Disallow /shop/InitiateShoppingListCmd
Disallow /shop/SaveSubstitutionPreferenceCmd
Disallow /shop/Logoff
Disallow /home/forum.html
Disallow /content/waitrose/en/home/forum/
Disallow /shop/MyAccount
Disallow /shop/MyAddresses
Disallow /shop/MyLists
Disallow /shop/GetCustomerOrders
Disallow /ecom/myaccount/my-orders*
Disallow /shop/BookSlot
Disallow /api/token-client-prod/v1/auth*
Disallow /shop/OfferDetails
Disallow /ecom/shop/offers
Disallow */undefined/*
Disallow *search?*
Disallow *HeaderSearchCmd*
Disallow *en/profile*
Disallow *A4.html*
Disallow *A5.html*
Disallow */webapp/*
Disallow *?fulfilment*
Disallow /shop/LogonForm*
Disallow *groceries/refined_by/search_term*
Disallow */customer-service/forms*
Disallow /ecom/bookslot/*
Disallow *%3Daff_awin*
Disallow *irclickid%3D*
Disallow *gclid*
Disallow /ecom/lists
Disallow /ecom/favourites
Disallow /ecom/for-you*
Disallow /ecom/quick-shop*
Disallow /ecom/my-details
Disallow /ecom/myaccount/my-orders
Disallow /ecom/shop/trolley
Disallow /ecom/checkout
Disallow /ecom/myaccount
Disallow /ecom/serviceselection
Disallow /mfe/*
Disallow *_offers/*
Allow /offers
Allow */offers/beer_wine_and_spirits_offers/beer_offers?*
Allow */offers/beer_wine_and_spirits_offers/champagne_and_sparkling_wine_offers/champagne_offers?*
Allow */offers/beer_wine_and_spirits_offers/champagne_and_sparkling_wine_offers/prosecco_offers?*
Allow */offers/beer_wine_and_spirits_offers/champagne_and_sparkling_wine_offers?*
Allow */offers/beer_wine_and_spirits_offers/spirits_and_liqueurs_offers/gin_offers?*
Allow */offers/beer_wine_and_spirits_offers/spirits_and_liqueurs_offers/whisky_offers?*
Allow */offers/beer_wine_and_spirits_offers/spirits_and_liqueurs_offers?*
Allow */offers/beer_wine_and_spirits_offers/wine_offers/fine_wine_offers?*
Allow */offers/beer_wine_and_spirits_offers/wine_offers/red_wine_offers?*
Allow */offers/beer_wine_and_spirits_offers/wine_offers/rose_wine_offers?*
Allow */offers/beer_wine_and_spirits_offers/wine_offers/white_wine_offers?*
Allow */offers/beer_wine_and_spirits_offers/wine_offers?*
Allow */offers/easter_offers*
Allow */offers/fresh_and_chilled_offers/fresh_meat_offers/lamb_offers?*
Allow */offers/fresh_and_chilled_offers/fresh_meat_offers?*
Allow */offers/fresh_and_chilled_offers/fresh_meat_offers/bbq_meat_offers?*
Allow */offers/frozen_offers/ice_cream_frozen_yogurt_and_sorbets_offers?*
Allow */offers/summer_offers/barbecue_offers?*
Allow */offers/summer_offers/picnic_offers?*
Allow *_offers$
Disallow /ajax/megamenu/lastupdate.json
Disallow *store%3D*
Disallow */https%3A/www.waitroseflorist.com*
Disallow */shop/buylist/*

Other Records

Field Value
crawl-delay 1

ia_archiver

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.waitrose.com/sitemapIndex.xml
Back to top

Comments

  • /robots.txt file for WR Fusion site
  • Version 1.4 2011-04-20
  • Trolley Pages
  • Check Out
  • User details
  • Mfe
  • offers directives
  • AJAX Megamenu JSON File
  • store= removals
  • valentines-drop-down
  • Buylist pages
  • Remove from internet archives
Back to top