Log In Sign Up

johnlewis.co.uk
robots.txt

Robots Exclusion Standard data for johnlewis.co.uk

Resource Scan

Scan Details

Site Domain johnlewis.co.uk
Base Domain johnlewis.co.uk
Scan Status Ok
Last Scan2024-11-17T01:13:08+00:00
Next Scan 2024-12-17T01:13:08+00:00

Last Scan

Scanned2024-11-17T01:13:08+00:00
URL http://johnlewis.co.uk/robots.txt
Redirect https://www.johnlewis.com/robots.txt
Redirect Domain www.johnlewis.com
Redirect Base johnlewis.com
Domain IPs 193.35.250.65, 193.35.254.65
Redirect IPs 96.17.180.42, 96.17.180.44
Response IP 96.17.180.44
Found Yes
Hash 37cf3fb8daa7c179adb62ffbd3cbbec99896526e0683ea43cada2c11d59fe8dd
SimHash 9844931ead26

Groups

*

Rule Path
Allow /browse/*/_/N-*
Allow /brand/*/_/N-*
Disallow */_/N-*
Disallow /menu
Disallow /content-renderer/
Allow */content-renderer/seo*
Disallow */write-a-review?productId*
Disallow */_/N-*Z*Z*Z*Z*Z*
Disallow /brand/*/N-*Z*Z*
Disallow /browse/edits/*/N-*Z*Z*
Disallow /myjohnlewis
Disallow /basket
Disallow /checkout
Disallow /sign-in
Disallow /create-account
Disallow /register$
Disallow /browse/_/
Disallow /browse/*/N-1y*
Disallow /browse/*/N-1z*
Disallow */_/N-bor*
Disallow */_/N-7hfd*
Disallow */_/N-a65*
Disallow */_/N-fje*
Disallow */_/N-e9n*
Disallow */_/N-a30*
Disallow */_/N-8b0*
Disallow */_/N-ejd*
Disallow *fabricSku%3D
Disallow *legType%3D
Allow /browse/*/N-*Z1z0ktrf$
Allow /brand/*/N-*Z1z0ktrf$
Allow /browse/*/N-2q3pZ1z0vwzuZ1z0ktrf$
Disallow /browse/*/N-*Z*Z1z0ktrf$
Disallow /brand/*/N-*Z*Z1z0ktrf$
Disallow */click-collect/
Disallow */collect%2B/
Disallow */express-delivery/
Disallow */by-approved-supplier/
Disallow */standard-uk-delivery/
Disallow */next-day-delivery/
Disallow */collect%2B-from-a-local-shop/
Disallow */4%2B/
Disallow */3-4/
Disallow */1-2/
Disallow */2-3/
Disallow */new-in-*
Disallow */new-in/
Allow /new-in/c300000007497
Allow /browse/baby-child/girlswear/new-in/_/N-1cu9
Allow /browse/women/womens-dresses/new-in/_/N-flwZ7lhg
Allow /browse/furniture-lights/bedroom/beds/new-in/_/N-c7oZ7lhg
Allow /browse/furniture-lights/bedroom/mattresses/view-all-mattresses/new-in/_/N-caoZ7lhg
Allow /browse/baby-child/shop-toys/view-all-toys/new-in/_/N-feuZ7lhg
Allow /browse/electricals/mobile-phones-accessories/view-all-mobile-phones/new-in/_/N-a8vZ7lhg
Allow /browse/men/mens-trainers/new-in/_/N-7oo1Z7lhg
Allow /browse/beauty/womens-fragrance/new-in/_/N-a63Z7lhg
Allow /brand/samsung/new-in/_/N-1z13z13Z7lhg
Allow /brand/microsoft/new-in/_/N-1z13ynnZ7lhg
Allow /brand/converse/new-in/_/N-1z1409fZ7lhg
Allow /brand/lego/new-in/_/N-1z140g1Z7lhg
Allow /brand/boden/new-in/_/N-1yzzr58Z7lhg
Allow /browse/women/womens-trainers-plimsolls/new-in/_/N-7oo6Z7lhg
Disallow */reduced-to-clear/
Disallow /browse/special-offers/electrical-offers/holding-pen/*
Disallow */discount%3D
Disallow */price%3D
Allow */price%3D%26
Disallow *-year-guarantee-included/
Disallow *-year-frame-guarantee/
Disallow */by-approved-supplier/
Disallow */gift-finder/*
Back to top

Other Records

Field Value
sitemap https://www.johnlewis.com/siteindex.xml
Back to top

Comments

  • 2024-11-11
  • Stop N Val Crawl without Browse or Brand
  • Random
  • Write a Review
  • Z Value Blocking
  • Account
  • Browse
  • over-indexing
  • Delivery Options
  • Allowing level 1 but blocking level 2 next day delivery pages
  • Average Review Rating
  • New In
  • On Offer
  • Discount Percentage
  • Price
  • Guarantees
  • Gift Finder
Back to top