hollandandbarrett.be
robots.txt

Robots Exclusion Standard data for hollandandbarrett.be

Resource Scan

Scan Details

Site Domain hollandandbarrett.be
Base Domain hollandandbarrett.be
Scan Status Ok
Last Scan2024-05-14T17:10:05+00:00
Next Scan 2024-06-13T17:10:05+00:00

Last Scan

Scanned2024-05-14T17:10:05+00:00
URL https://hollandandbarrett.be/robots.txt
Redirect https://www.hollandandbarrett.be/robots.txt
Redirect Domain www.hollandandbarrett.be
Redirect Base hollandandbarrett.be
Domain IPs 23.52.171.201, 23.59.168.155, 2600:1413:b000:1b::17d7:707, 2600:1413:b000:1b::17d7:71e
Redirect IPs 23.52.171.219, 23.59.168.177, 2600:1413:b000:1b::17d7:707, 2600:1413:b000:1b::17d7:71f
Response IP 23.52.40.9
Found Yes
Hash 552ca011a6921185afc800a61ca7dbcbe05b1a9cfb9f5ba98f6142804b985837
SimHash 09deebe3d690

Groups

*

Rule Path
Disallow /basket/
Disallow /checkout/
Disallow /common/
Disallow /modules/
Disallow /auth/
Disallow /template/
Disallow /emailtemplates/
Disallow /referentiemateriaal
Disallow /api/pdp/images*
Disallow /api/plp/global-layout*
Disallow /api/social-messaging/
Disallow /shop/product/streams/media/
Disallow /rest/api/basket*
Disallow /rest/api/session*
Disallow /*browse*
Disallow /*rating_*
Disallow /*gender_*
Disallow /*price_*
Disallow /*size_*
Disallow /*strength-mg*
Disallow /*strength-iu*
Disallow /*showAll%3D*
Disallow /*skupromo_*
Disallow /*strength-g_less-than*
Disallow /*strength-c_less-than*
Disallow /*strength-ml_less-than*
Disallow /*sort_ratings*
Disallow /*sort_high-low*
Disallow /*sort_low-high*
Disallow /*sort_a-to-z*
Disallow /*sort_z-to-a*
Disallow /*sort_topseller*
Disallow /*N%3D*
Disallow /_next/static*
Disallow /*brandId%3D*
Disallow /*bvroute%3D*
Disallow /*bvstate%3D*
Disallow /*cat%3D*
Disallow /*isOneSearchMVT%3D*
Disallow /*limit%3D*
Disallow /*page-sort%3D*
Disallow /*SID%3D*
Allow /*.js
Allow /*.css
Allow /*.jpg
Allow /*.jpeg
Allow /*.svg
Allow /*.gif
Allow /*.png
Allow /*.js*
Allow /*.css*
Allow /*.jpg*
Allow /*.jpeg*
Allow /*.svg*
Allow /*.gif*
Allow /*.png*
Allow /_next/static/*.js
Allow /_next/static/*.css

adidxbot

Rule Path
Allow /

adsbot-google-mobile

Rule Path
Allow /

adsbot-google

Rule Path
Allow /

Other Records

Field Value
sitemap https://www.hollandandbarrett.be/sitemap.xml

Comments

  • sitemaps
  • Crawlers Setup
  • Paths (no clean URLs)
  • Disallow: /search/
  • Disallow: /*isSearch=*
  • Disallow: /*query=*
  • allow all resources
  • Website Sitemap