westburne.ca
robots.txt

Robots Exclusion Standard data for westburne.ca

Resource Scan

Scan Details

Site Domain westburne.ca
Base Domain westburne.ca
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-11-11T15:45:35+00:00
Next Scan 2026-01-10T15:45:35+00:00

Last Successful Scan

Scanned2025-09-06T00:30:54+00:00
URL https://westburne.ca/robots.txt
Redirect https://www.westburne.ca/robots.txt
Redirect Domain www.westburne.ca
Redirect Base westburne.ca
Domain IPs 13.107.253.59
Redirect IPs 13.107.246.59, 2620:1ec:bdf::59
Response IP 13.107.246.59
Found Yes
Hash be3c9d2656578f6bdc7c2b6d6a272a6fbac51db26f6330da87679d6109372693
SimHash 95425a636e10

Groups

*

Rule Path
Disallow */cwr/cart
Disallow */cwr/cart/*
Disallow *_%3D*
Disallow */*/c/brands
Disallow *//c/
Disallow */_s/saveFulfillmentJson
Disallow */addBulkProductItemsToCart
Disallow */addBulkProductItemsToCartReplacementModel
Disallow */addCustomerUserNboProductTocart
Disallow */addObseleteReplacementProductItemsToCart
Disallow */addProductsToCartFormConfigurator
Disallow */addToCartFromUrl
Disallow */addtoJoblist/*
Disallow */c/.seo-collapse
Disallow */**/c/browseCategoryAndBrandForMobile
Disallow */checkout
Disallow */checkout/*
Disallow */closeGlobalPromotion
Disallow */closeNotificationIcon
Disallow *collapseTwo
Disallow */combined.css*
Disallow */combined.js*
Disallow */compare/
Disallow */cookieCosent
Disallow */cookieCosentToggler
Disallow */enableCookieConsent
Disallow */erp/*
Disallow */fast-register
Disallow */findDefaultStore
Disallow */findNearestStoreForAnonymous
Disallow */footer/globalSites
Disallow */GDPRCookieContainerComponentController/*
Disallow */getBranchList*
Disallow */getHtmlFormContent
Disallow */getInspirationCompanyUrl
Disallow */getJWTTokenAndUserDetails
Disallow */getLimitedDistributionPickupZip
Disallow */getNotificationMsgvaluesForLD
Disallow */isQuantityVisibleForProducts
Disallow */oauth/getOauthToken
Disallow */getPagablePickupBranchesForAnonymous.json
Disallow */getRegionList
Disallow */getReplacementForObsoleteproduct
Disallow */getTieredPricingMap
Disallow */isQuantitySectionVisible
Disallow */login
Disallow */login/*
Disallow */logout
Disallow */mergeCart/*
Disallow */mobilelogin*
Disallow */mobileprivacy
Disallow */mobileRegistration*
Disallow */mockMedia/*
Disallow */my-account
Disallow */my-account/*
Disallow */myaccountpopup
Disallow */my-company/*
Disallow */mysettings
Disallow */notification/*
Disallow */oauth/getOauthToken
Disallow */obseleteProductAddToCart
Disallow */order-return/*
Disallow */order-new-return/*
Disallow */p/getEmailPDPpopup
Disallow */p/getTieredPricing
Disallow */p/slangKeyword
Disallow *page%3D0
Disallow *page%3D-1
Disallow *pageSize%3D*%26
Disallow */prefillAddToCart
Disallow */priceToolTip/*
Disallow */product360view/*
Disallow */productBranchStock
Disallow */productBranchStock/*
Disallow */quickorder
Disallow */recommendedCategories.ajax
Disallow */register
Disallow */register/*
Disallow */removeChangeRegionPopup
Disallow */removeGDPRLegalNotification
Disallow */removePCIComplianceNotification
Disallow */replacementProduct
Disallow */sales/*
Disallow */updatelanguage
Disallow */updateRegion
Disallow */validateUpdateRegion
Disallow */view/TabbedNavigationComponentController/updateTabIndex
Disallow */p/*/zoomImages
Disallow *%7B0%7D
Disallow *%7B0%7D
Disallow */cms/getPDPContent.json?productCode=*
Disallow */_ui/addons/*
Disallow */_ui/mobile/common/*
Disallow */_ui/resonsive/common/js/*
Disallow */perse/getNoResultsPage
Disallow */search*
Disallow */_ui/angular/*
Allow *%3A*%3A*/p/*
Allow */p/*/*
Disallow /search*.com
Disallow /search*.net
Disallow */medias/*
Disallow *%3Arelevance%3A*%3A*
Disallow *%3Arelevance%3A*%3A*
Disallow *%3A*%3A*
Disallow *%3A*%3A*
Disallow */c/*__*
Disallow */c/*/*productbranchstock*
Disallow *page%3D-1*
Disallow */c/*?q=%3Arelevance&text=&filter=&aliasname=&partialSearch=
Disallow */Category/*/c/*relevance%3Acategory*
Disallow */Brand/*/c/*relevance%3Abrand*
Allow */Category/*/c/*q%3D%3Arelevance%3Abrand%3A*
Disallow */Category/*/c/*q%3D%3Arelevance%3Abrand%3A*%3A
Disallow */Category/*/c/*q%3D%3Arelevance%3Abrand%3A*page%3D-1*
Allow */Brand/*/c/*q%3D%3Arelevance%3Acategory%3A*
Disallow */Brand/*/c/*q%3D%3Arelevance%3Acategory%3A*%3A
Disallow */Brand/*/c/*q%3D%3Arelevance%3Acategory%3A*page%3D-1*

scrapy

Rule Path
Disallow /

cazoodlebot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot/1.0

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

aspiegelbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.westburne.ca/cwr/sitemap.xml

Comments

  • cms related
  • testing
  • spam pattern
  • site specific blocks
  • Limit PLP Facet Crawling
  • Disallow facets but those allowed elsewhere
  • disallow other
  • disallow facet removal
  • disallow category facet on category pages
  • Disallow brand facet on brand pages
  • Allow / Disallow Brand facets for category pages
  • Allow / Disallow Category Facets for Brand pages
  • v20231017