waitrose.com
robots.txt

Robots Exclusion Standard data for waitrose.com

Resource Scan

Scan Details

Site Domain waitrose.com
Base Domain waitrose.com
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2025-11-23T00:06:11+00:00
Next Scan 2026-01-22T00:06:11+00:00

Last Successful Scan

Scanned2025-08-28T19:39:16+00:00
URL https://waitrose.com/robots.txt
Redirect https://www.waitrose.com/robots.txt
Redirect Domain www.waitrose.com
Redirect Base waitrose.com
Domain IPs 23.50.88.178
Redirect IPs 23.50.88.178
Response IP 23.199.143.13
Found Yes
Hash 13c71439b45d930f5d7c2a434321a31608a459e44b148f9d7a93a3b10e7c0cb4
SimHash 6d363d5f1ab4

Groups

*

Rule Path
Disallow /shop/ViewTrolley
Disallow /shop/RegistrationInitiate
Disallow /shop/CheckOutInitiateCmd
Disallow /shop/CheckOutOrderPreviewCmd
Disallow /shop/CheckOutPaymentCmd
Disallow /shop/OrderProcess
Disallow /shop/InitiateLogin
Disallow /shop/RegistrationInitiate
Disallow /shop/InitiateMyAccount
Disallow /shop/LoginDetailsInitiateCmd
Disallow /shop/PrivateInfoInitiateCmd
Disallow /shop/AddressInitiateCmd
Disallow /shop/GetPaymentBillingDetailsCmd
Disallow /shop/OrderStatusDisplayCmd
Disallow /shop/InterestItemDisplayCmd
Disallow /shop/InitiateShoppingListCmd
Disallow /shop/SaveSubstitutionPreferenceCmd
Disallow /shop/Logoff
Disallow /home/forum.html
Disallow /content/waitrose/en/home/forum/
Disallow /shop/MyAccount
Disallow /shop/MyAddresses
Disallow /shop/MyLists
Disallow /shop/GetCustomerOrders
Disallow /ecom/myaccount/my-orders*
Disallow /shop/BookSlot
Disallow /api/token-client-prod/v1/auth*
Disallow /shop/OfferDetails
Disallow /ecom/shop/offers
Disallow */undefined/*
Disallow *search?*
Disallow *HeaderSearchCmd*
Disallow *en/profile*
Disallow *A4.html*
Disallow *A5.html*
Disallow */webapp/*
Disallow *?fulfilment*
Disallow /shop/LogonForm*
Disallow *groceries/refined_by/search_term*
Disallow */customer-service/forms*
Disallow /ecom/bookslot/*
Disallow *%3Daff_awin*
Disallow *irclickid%3D*
Disallow *gclid*
Disallow /ecom/lists
Disallow /ecom/favourites
Disallow /ecom/for-you*
Disallow /ecom/quick-shop*
Disallow /ecom/my-details
Disallow /ecom/myaccount/my-orders
Disallow /ecom/shop/trolley
Disallow /ecom/checkout
Disallow /ecom/myaccount
Disallow /ecom/serviceselection
Disallow /mfe/*
Disallow *_offers/*
Allow /offers
Allow */offers/beer_wine_and_spirits_offers/beer_offers?*
Allow */offers/beer_wine_and_spirits_offers/champagne_and_sparkling_wine_offers/champagne_offers?*
Allow */offers/beer_wine_and_spirits_offers/champagne_and_sparkling_wine_offers/prosecco_offers?*
Allow */offers/beer_wine_and_spirits_offers/champagne_and_sparkling_wine_offers?*
Allow */offers/beer_wine_and_spirits_offers/spirits_and_liqueurs_offers/gin_offers?*
Allow */offers/beer_wine_and_spirits_offers/spirits_and_liqueurs_offers/whisky_offers?*
Allow */offers/beer_wine_and_spirits_offers/spirits_and_liqueurs_offers?*
Allow */offers/beer_wine_and_spirits_offers/wine_offers/fine_wine_offers?*
Allow */offers/beer_wine_and_spirits_offers/wine_offers/red_wine_offers?*
Allow */offers/beer_wine_and_spirits_offers/wine_offers/rose_wine_offers?*
Allow */offers/beer_wine_and_spirits_offers/wine_offers/white_wine_offers?*
Allow */offers/beer_wine_and_spirits_offers/wine_offers?*
Allow */offers/easter_offers*
Allow */offers/fresh_and_chilled_offers/fresh_meat_offers/lamb_offers?*
Allow */offers/fresh_and_chilled_offers/fresh_meat_offers?*
Allow */offers/fresh_and_chilled_offers/fresh_meat_offers/bbq_meat_offers?*
Allow */offers/frozen_offers/ice_cream_frozen_yogurt_and_sorbets_offers?*
Allow */offers/summer_offers/barbecue_offers?*
Allow */offers/summer_offers/picnic_offers?*
Allow *_offers$
Disallow /ajax/megamenu/lastupdate.json
Disallow *store%3D*
Disallow */https%3A/www.waitroseflorist.com*
Disallow */shop/buylist/*

Other Records

Field Value
crawl-delay 1

ia_archiver

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.waitrose.com/sitemapIndex.xml

Comments

  • /robots.txt file for WR Fusion site
  • Version 1.4 2011-04-20
  • Trolley Pages
  • Check Out
  • User details
  • Mfe
  • offers directives
  • AJAX Megamenu JSON File
  • store= removals
  • valentines-drop-down
  • Buylist pages
  • Remove from internet archives