Log In Sign Up

harrods.com
robots.txt

Robots Exclusion Standard data for harrods.com

Resource Scan

Scan Details

Site Domain harrods.com
Base Domain harrods.com
Scan Status Ok
Last Scan2025-12-07T03:21:05+00:00
Next Scan 2025-12-21T03:21:05+00:00

Last Scan

Scanned2025-12-07T03:21:05+00:00
URL https://harrods.com/robots.txt
Redirect https://www.harrods.com:443/robots.txt
Redirect Domain www.harrods.com
Redirect Base harrods.com
Domain IPs 104.18.140.105, 104.18.141.105
Redirect IPs 104.18.35.42, 172.64.152.214, 2606:4700:440b::ac40:98d6, 2a06:98c1:310d::6812:232a
Response IP 172.64.152.214
Found Yes
Hash afa1d1cdebcc7817ed3503cb6c78e9321100aa9ef1f7aae107b928a3a8c74e1a
SimHash 7ab4140bd19b

Groups

pinterestbot
meta-external-agent
amazonbot
facebookexternalhit
petalbot
claudebot

Rule Path
Disallow /*?*=*

Other Records

Field Value
crawl-delay 10

*

Rule Path
Disallow */account$
Disallow */login$
Disallow */basket$
Disallow */pre-checkout$
Disallow */checkout$
Disallow */admin$
Disallow */wishlist$
Disallow *000000000007827210*
Disallow *000000000007852421*
Disallow /search
Disallow /*?*=*
Allow /*?page=
Disallow /en-ad/
Disallow /en-af/
Disallow /en-ag/
Disallow /en-ai/
Disallow /en-an/
Disallow /en-aq/
Disallow /en-aw/
Disallow /en-bb/
Disallow /en-bf/
Disallow /en-bi/
Disallow /en-bj/
Disallow /en-bo/
Disallow /en-bv/
Disallow /en-bw/
Disallow /en-bz/
Disallow /en-cc/
Disallow /en-cf/
Disallow /en-cg/
Disallow /en-ck/
Disallow /en-cv/
Disallow /en-cx/
Disallow /en-dj/
Disallow /en-dm/
Disallow /en-eh/
Disallow /en-fj/
Disallow /en-fk/
Disallow /en-fm/
Disallow /en-fo/
Disallow /en-ga/
Disallow /en-gd/
Disallow /en-gf/
Disallow /en-gg/
Disallow /en-gi/
Disallow /en-gl/
Disallow /en-gm/
Disallow /en-gn/
Disallow /en-gp/
Disallow /en-gq/
Disallow /en-gu/
Disallow /en-gw/
Disallow /en-gy/
Disallow /en-hm/
Disallow /en-hn/
Disallow /en-ht/
Disallow /en-ic/
Disallow /en-io/
Disallow /en-iq/
Disallow /en-je/
Disallow /en-kg/
Disallow /en-ki/
Disallow /en-km/
Disallow /en-kn/
Disallow /en-kv/
Disallow /en-ky/
Disallow /en-la/
Disallow /en-lc/
Disallow /en-ls/
Disallow /en-mh/
Disallow /en-ml/
Disallow /en-mp/
Disallow /en-mq/
Disallow /en-ms/
Disallow /en-mv/
Disallow /en-mw/
Disallow /en-na/
Disallow /en-nc/
Disallow /en-ne/
Disallow /en-nf/
Disallow /en-nr/
Disallow /en-nu/
Disallow /en-pf/
Disallow /en-pg/
Disallow /en-pm/
Disallow /en-pn/
Disallow /en-pw/
Disallow /en-re/
Disallow /en-rw/
Disallow /en-sb/
Disallow /en-sc/
Disallow /en-sh/
Disallow /en-sl/
Disallow /en-so/
Disallow /en-sr/
Disallow /en-st/
Disallow /en-sz/
Disallow /en-tc/
Disallow /en-td/
Disallow /en-tg/
Disallow /en-tj/
Disallow /en-tk/
Disallow /en-tl/
Disallow /en-to/
Disallow /en-tv/
Disallow /en-va/
Disallow /en-vc/
Disallow /en-vg/
Disallow /en-vi/
Disallow /en-vu/
Disallow /en-wf/
Disallow /en-ye/
Disallow /en-xc/
Disallow /en-zr/
Disallow /en-by/
Disallow /en-ci/
Disallow /en-sn/
Disallow /en-sv/
Disallow /en-et/
Disallow /en-py/
Disallow /en-mg/
Disallow /en-ni/
Disallow /en-li/
Disallow /en-bt/
Disallow /en-sm/
Disallow /en-mz/
Disallow /en-uy/
Disallow /en-cm/
Disallow /en-as/
Disallow /en-tf/
Disallow /en-im/
Disallow /en-lr/
Disallow /en-mm/
Disallow /en-sd/
Disallow /en-sj/
Disallow /en-tm/
Disallow /en-um/
Disallow /en-sy/
Back to top

Other Records

Field Value
sitemap https://harrods.com/sitemap_index.xml
sitemap https://harrods.com/legacy_sitemap_index.xml
Back to top

Comments

  • Targeted Bot Restrictions
  • Personalised Routes
  • URL Paths
  • Block Parameters
  • Allow Override For Pagination
  • Subfolders
  • ROOT SITEMAP FOR BOTS TO INDEX
Back to top