hollandandbarrett.com
robots.txt

Robots Exclusion Standard data for hollandandbarrett.com

Resource Scan

Scan Details

Site Domain hollandandbarrett.com
Base Domain hollandandbarrett.com
Scan Status Ok
Last Scan2024-11-04T11:42:51+00:00
Next Scan 2024-11-18T11:42:51+00:00

Last Scan

Scanned2024-11-04T11:42:51+00:00
URL https://hollandandbarrett.com/robots.txt
Redirect https://www.hollandandbarrett.com/robots.txt
Redirect Domain www.hollandandbarrett.com
Redirect Base hollandandbarrett.com
Domain IPs 2600:1413:b000:1b::17d7:707, 2600:1413:b000:1b::17d7:71f, 96.17.180.48, 96.17.180.50
Redirect IPs 23.59.168.96, 2600:1413:b000:1b::17d7:707, 2600:1413:b000:1b::17d7:71f
Response IP 184.28.235.66
Found Yes
Hash e36363539aa0f4060282813510a968f85eab779355875dc7cd36be89be8ea6bb
SimHash 091c6bc3d790

Groups

*

Rule Path
Allow /*sort_new-in*
Disallow /v1/widget*
Disallow /basket/
Disallow /checkout/
Disallow /common/
Disallow /modules/
Disallow /template/
Disallow /emailtemplates/
Disallow /healthbox/
Disallow /my-account/
Disallow /auth/
Disallow /cartridges/
Disallow /*shop/gs/*
Disallow /*en-us/shop/gs/*
Disallow /*en-au/shop/gs/*
Disallow /*search/?isSearch=true&query=*
Disallow /*gclid%3D*
Disallow /*dclid%3D*
Disallow /*page-sort%3D*
Disallow /*browse*
Disallow /*rating_*
Disallow /*gender_*
Disallow /*price_*
Disallow /*size_*
Disallow /*strength-mg*
Disallow /*strength-iu*
Disallow /*showAll%3D*
Disallow /*skupromo_*
Disallow /*strength-c_less-than*
Disallow /*sort_ratings*
Disallow /*sort_high-low*
Disallow /*sort_low-high*
Disallow /*sort_a-to-z*
Disallow /*sort_z-to-a*
Disallow /*sort_topseller*
Disallow /*N%3D*
Disallow /*q%3D*
Disallow /*bvstate*
Disallow /*bvroute*
Disallow /_next/static*
Disallow /*query*
Disallow /*isSearch*
Disallow /*isOneSearchMVT*
Allow /*.js
Allow /*.css
Allow /*.jpg
Allow /*.jpeg
Allow /*.svg
Allow /*.gif
Allow /*.png
Allow /*.js*
Allow /*.css*
Allow /*.jpg*
Allow /*.jpeg*
Allow /*.svg*
Allow /*.gif*
Allow /*.png*

adidxbot

Rule Path
Allow /

adsbot-google-mobile

Rule Path
Allow /

adsbot-google

Rule Path
Allow /

Other Records

Field Value
sitemap https://www.hollandandbarrett.com/sitemap.xml

Comments

  • Website Sitemap
  • Crawlers Setup
  • Paths (no clean URLs)
  • allow all resources