Log In Sign Up

rexel.fr
robots.txt

Robots Exclusion Standard data for rexel.fr

Resource Scan

Scan Details

Site Domain rexel.fr
Base Domain rexel.fr
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-11-14T22:19:35+00:00
Next Scan 2024-11-28T22:19:35+00:00

Last Successful Scan

Scanned2024-10-23T22:18:51+00:00
URL https://rexel.fr/robots.txt
Redirect https://www.rexel.fr/robots.txt
Redirect Domain www.rexel.fr
Redirect Base rexel.fr
Domain IPs 13.107.246.59
Redirect IPs 13.107.246.59, 2620:1ec:bdf::59
Response IP 13.107.246.59
Found Yes
Hash 740975eb0dc1357c23a0b067b2f748ec45e7e781ef31cd996c82ebd4a5de502e
SimHash 51825a376a50

Groups

*

Rule Path Comment
Disallow */frx/cart -
Disallow */frx/cart/* -
Disallow *_%3D* -
Disallow */*/c/brands -
Disallow *//c/ -
Disallow */_s/saveFulfillmentJson -
Disallow */addBulkProductItemsToCart -
Disallow */addBulkProductItemsToCartReplacementModel -
Disallow */addCustomerUserNboProductTocart -
Disallow */addObseleteReplacementProductItemsToCart -
Disallow */addProductsToCartFormConfigurator -
Disallow */addToCartFromUrl -
Disallow */addtoJoblist/* -
Disallow */c/.seo-collapse -
Disallow */**/c/browseCategoryAndBrandForMobile -
Disallow */checkout -
Disallow */checkout/* -
Disallow */closeGlobalPromotion -
Disallow */closeNotificationIcon -
Disallow *collapseTwo -
Disallow */combined.css* -
Disallow */combined.js* -
Disallow */compare/ -
Disallow */cookieCosent -
Disallow */cookieCosentToggler -
Disallow */deliveryDateSimulatorData.json -
Disallow */enableCookieConsent -
Disallow */erp/* -
Disallow */fast-register -
Disallow */findDefaultStore -
Disallow */findNearestStoreForAnonymous -
Disallow */footer/globalSites -
Disallow */GDPRCookieContainerComponentController/* -
Disallow */getBranchList* -
Disallow */getHtmlFormContent -
Disallow */getInspirationCompanyUrl -
Disallow */getJWTTokenAndUserDetails -
Disallow */getLimitedDistributionPickupZip -
Disallow */getNotificationMsgvaluesForLD -
Disallow */isQuantityVisibleForProducts -
Disallow */oauth/getOauthToken -
Disallow */getHtmlFormContent -
Disallow */getPagablePickupBranchesForAnonymous.json -
Disallow */getRegionList -
Disallow */getReplacementForObsoleteproduct -
Disallow */getTieredPricingMap -
Disallow */getJWTTokenAndUserDetails -
Disallow */isQuantitySectionVisible -
Disallow */isQuantityVisibleForProducts -
Disallow */login -
Disallow */login/* -
Disallow */logout -
Disallow */medias/*.pdf -
Disallow */mergeCart/* -
Disallow */mobilelogin* -
Disallow */mobileprivacy -
Disallow */mobileRegistration* -
Disallow */mockMedia/* -
Disallow */mostordered/getcategories -
Disallow */my-account -
Disallow */my-account/* -
Disallow */myaccountpopup -
Disallow */my-company/* -
Disallow */mysettings -
Disallow */newQuickorder/* -
Disallow */notification/* -
Disallow */oauth/getOauthToken -
Disallow */obseleteProductAddToCart -
Disallow */order-return/* -
Disallow */order-new-return/* -
Disallow */p/getEmailPDPpopup -
Disallow */p/getTieredPricing -
Disallow */p/slangKeyword -
Disallow */pf/getProductSelectedVariant -
Disallow */pf/productvariants -
Disallow *page%3D0 -
Disallow *page%3D-1 -
Disallow *pageSize%3D*%26 -
Disallow */perse/getNoResultsPage -
Disallow */prefillAddToCart -
Disallow */priceToolTip/* -
Disallow */product360view/* -
Disallow */productBranchStock -
Disallow */productBranchStock/* -
Disallow */punchoutCxmlCart/punchout -
Disallow */quickorder -
Disallow */recommendedCategories.ajax -
Disallow */register -
Disallow */register/* -
Disallow */removeChangeRegionPopup -
Disallow */removeGDPRLegalNotification -
Disallow */removePCIComplianceNotification -
Disallow */replacementProduct -
Disallow */sales/* -
Disallow */search/* -
Disallow */search* -
Disallow */search/searchCategory -
Disallow */updatelanguage -
Disallow */updateRegion -
Disallow */validateUpdateRegion -
Disallow */view/TabbedNavigationComponentController/updateTabIndex -
Disallow */p/*/zoomImages -
Disallow *%7B0%7D -
Disallow *%7B0%7D -
Disallow */cms/getPDPContent.json?productCode=* -
Allow *%3A*%3A*/p/* -
Disallow /search*.com -
Disallow /search*.net -
Disallow */schneider_scconfigurator -
Disallow *isOnlyProductIds%3D%26isOrderForm%3D%26skuIndex%3D%26isCreateOrderForm* -
Disallow */c/*displayProductList* -
Disallow */Cat%C3%A9gorie/*/c/*q%3D%3Arelevance*pageSize%3D10* -
Disallow */Cat%C3%A9gorie/*/c/*q%3D%3Arelevance*pageSize%3D12* -
Disallow */Cat%C3%A9gorie/*/c/*q%3D%3Arelevance*pageSize%3D10* -
Disallow */Cat%C3%A9gorie/*/c/*q%3D%3Arelevance*pageSize%3D12* -
Disallow */Fabricant/*/c/*q%3D%3Arelevance*pageSize%3D10* -
Disallow */Fabricant/*/c/*q%3D%3Arelevance*pageSize%3D12* -
Disallow */Fabricant/*/c/*q%3D%3Arelevance*pageSize%3D10* -
Disallow */Fabricant/*/c/*q%3D%3Arelevance*pageSize%3D12* -
Disallow *relevance*%26filter%3D*%3A* old + new
Disallow *relevance%26filter%3D*%3A* old + new
Disallow *filter%3D*%3A* -
Disallow *filter%3D*%3A*~* -
Disallow *filter%3D*%3A*%3B*%3A* -
Allow *filter%3DbrandCode%3A* -
Disallow *filter%3DbrandCode%3A*~ -
Disallow *filter%3DbrandCode%3A*%3B* -
Disallow */Fabricant/*/c/*filter%3DbrandCode%3A* -
Allow */Fabricant/*/c/*filter%3DcategoryCode%3A* -
Disallow */Fabricant/*/c/*filter%3DcategoryCode%3A*~ -
Disallow */Fabricant/*/c/*filter%3DcategoryCode%3A*%3B* -
Allow */Fabricant/*/c/*filter%3DclassificationName%3A* -
Disallow */Fabricant/*/c/*filter%3DclassificationName%3A*~ -
Disallow */Fabricant/*/c/*filter%3DclassificationName%3A*%3B* -
Allow */Fabricant/*/c/*filter%3DWF000001%3A* -
Disallow */Fabricant/*/c/*filter%3DWF000001%3A*~ -
Disallow */Fabricant/*/c/*filter%3DWF000001%3A*%3B* -
Disallow *main-*.js -
Disallow *scripts-*.js -
Disallow *polyfills-*.js -
Disallow *runtime-*.js -
Disallow *%3Arelevance%3A*%3A* -
Disallow *%3Arelevance%3A*%3A* -
Disallow */c/*__* -
Disallow */c/*/*productbranchstock* -
Disallow *page%3D-1* -
Disallow */c/*?q=%3Arelevance&text=&filter=&aliasname=&partialSearch= -
Disallow */c/*?q=%3Arelevance&text=&filter=&aliasname=&partialSearch= -
Disallow */Cat%C3%A9gorie/*/c/*relevance%3Acategory* -
Disallow */Cat%C3%A9gorie/*/c/*relevance%3Acategory* -
Disallow */Fabricant/*/c/*relevance%3AbrandCategoryIdName* -
Disallow */Fabricant/*/c/*relevance%3AbrandCategoryIdName* -
Disallow */Fabricant/*/c/*relevance%3AbrandCategoryIdName*pageSize%3D10* -
Disallow */Fabricant/*/c/*relevance%3AbrandCategoryIdName*pageSize%3D10* -
Disallow */Fabricant/*/c/*relevance%3AbrandCategoryIdName*pageSize%3D12* -
Disallow */Fabricant/*/c/*relevance%3AbrandCategoryIdName*pageSize%3D12* -
Disallow */Fabricant/*/c/*q%3D%3Arelevance%3Acategory%3A*pageSize%3D10* -
Disallow */Fabricant/*/c/*q%3D%3Arelevance%3Acategory%3A*pageSize%3D12* -
Disallow */Cat%C3%A9gorie/*/c/*%3Arelevance%3A*%3A*%3A*%3A* -
Disallow */Cat%C3%A9gorie/*/c/*%3Arelevance%3A*%3A*%3A*%3A* -
Disallow */Fabricant/*/c/*%3Arelevance%3A*%3A*%3A*3A* -
Disallow */Fabricant/*/c/*%3Arelevance%3A*%3A*%3A*%3A* -
Allow */Cat%C3%A9gorie/*/c/*q%3D%3Arelevance%3AbrandCategoryIdName%3A* -
Allow */Cat%C3%A9gorie/*/c/*q%3D%3Arelevance%3Abrand%3A* -
Allow */Cat%C3%A9gorie/*/c/*q%3D%3Arelevance%3AbrandCategoryIdName%3A* -
Allow */Cat%C3%A9gorie/*/c/*q%3D%3Arelevance%3Abrand%3A* -
Disallow */Cat%C3%A9gorie/*/c/*q%3D%3Arelevance%3AbrandCategoryIdName%3A*%3A -
Disallow */Cat%C3%A9gorie/*/c/*q%3D%3Arelevance%3AbrandCategoryIdName%3A*%3A -
Disallow */Cat%C3%A9gorie/*/c/*q%3D%3Arelevance%3AbrandCategoryIdName%3A*page%3D-1* -
Disallow */Cat%C3%A9gorie/*/c/*q%3D%3Arelevance%3AbrandCategoryIdName%3A*page%3D-1* -
Allow */Fabricant/*/c/*q%3D%3Arelevance%3Acategory%3A* -
Allow */Fabricant/*/c/*q%3D%3Arelevance%3Acategory%3A* -
Disallow */Fabricant/*/c/*q%3D%3Arelevance%3Acategory%3A*pageSize%3D10* -
Disallow */Fabricant/*/c/*q%3D%3Arelevance%3Acategory%3A*pageSize%3D12* -
Disallow */Fabricant/*/c/*q%3D%3Arelevance%3Acategory%3A*%3A -
Disallow */Fabricant/*/c/*q%3D%3Arelevance%3Acategory%3A*%3A -
Disallow */Fabricant/*/c/*q%3D%3Arelevance%3Acategory%3A*page%3D-1* -
Disallow */Fabricant/*/c/*q%3D%3Arelevance%3Acategory%3A*page%3D-1* -

bytespider

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

cazoodlebot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot/1.0

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

aspiegelbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

gptbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.rexel.fr/frx/sitemap.xml
Back to top

Comments

  • cms related
  • testing
  • allow odd PDP combinations
  • spam pattern
  • site specific blocks
  • old pagination facets - params - tests
  • PERSE
  • Limit PLP Facet Crawling
  • Disallow facets but those allowed elsewhere
  • disallow other
  • disallow facet removal
  • disallow category facet on category pages
  • Disallow brand facet on brand pages
  • disallow multi facet selections on category and brand
  • Allow / Disallow Brand facets for category pages
  • Allow / Disallow Category Facets for Brand pages
  • v20240523
Back to top