Log In Sign Up

rexel.de
robots.txt

Robots Exclusion Standard data for rexel.de

Resource Scan

Scan Details

Site Domain rexel.de
Base Domain rexel.de
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-11-11T13:19:36+00:00
Next Scan 2025-02-09T13:19:36+00:00

Last Successful Scan

Scanned2024-07-08T04:12:27+00:00
URL https://rexel.de/robots.txt
Redirect https://www.rexel.de/robots.txt
Redirect Domain www.rexel.de
Redirect Base rexel.de
Domain IPs 13.107.253.58
Redirect IPs 13.107.246.59, 2620:1ec:bdf::59
Response IP 13.107.246.59
Found Yes
Hash d457a2b0f42d40bb328b24fbe78c1979fce232ba098da9f43206106692f52daa
SimHash c1035a576a10

Groups

*

Rule Path
Disallow */cart
Disallow */cart/*
Disallow *_%3D*
Disallow */*/c/brands
Disallow *//c/
Disallow */_s/saveFulfillmentJson
Disallow */addBulkProductItemsToCart
Disallow */addBulkProductItemsToCartReplacementModel
Disallow */addCustomerUserNboProductTocart
Disallow */addObseleteReplacementProductItemsToCart
Disallow */addProductsToCartFormConfigurator
Disallow */addToCartFromUrl
Disallow */addtoJoblist/*
Disallow */c/.seo-collapse
Disallow */**/c/browseCategoryAndBrandForMobile
Disallow */checkout
Disallow */checkout/*
Disallow */closeGlobalPromotion
Disallow */closeNotificationIcon
Disallow *collapseTwo
Disallow */combined.css*
Disallow */combined.js*
Disallow */compare/
Disallow */cookieCosent
Disallow */cookieCosentToggler
Disallow */deliveryDateSimulatorData.json
Disallow */enableCookieConsent
Disallow */erp/*
Disallow */fast-register
Disallow */findDefaultStore
Disallow */findNearestStoreForAnonymous
Disallow */footer/globalSites
Disallow */GDPRCookieContainerComponentController/*
Disallow */getBranchList*
Disallow */getHtmlFormContent
Disallow */getInspirationCompanyUrl
Disallow */getJWTTokenAndUserDetails
Disallow */getLimitedDistributionPickupZip
Disallow */getNotificationMsgvaluesForLD
Disallow */isQuantityVisibleForProducts
Disallow */oauth/getOauthToken
Disallow */getHtmlFormContent
Disallow */getPagablePickupBranchesForAnonymous.json
Disallow */getRegionList
Disallow */getReplacementForObsoleteproduct
Disallow */getTieredPricingMap
Disallow */getJWTTokenAndUserDetails
Disallow */isQuantitySectionVisible
Disallow */isQuantityVisibleForProducts
Disallow */login
Disallow */login/*
Disallow */logout
Disallow */mergeCart/*
Disallow */mobilelogin*
Disallow */mobileprivacy
Disallow */mobileRegistration*
Disallow */mockMedia/*
Disallow */mostordered/getcategories
Disallow */my-account
Disallow */my-account/*
Disallow */myaccountpopup
Disallow */my-company/*
Disallow */mysettings
Disallow */newQuickorder/*
Disallow */notification/*
Disallow */oauth/getOauthToken
Disallow */obseleteProductAddToCart
Disallow */order-return/*
Disallow */order-new-return/*
Disallow */p/getEmailPDPpopup
Disallow */p/getTieredPricing
Disallow */p/slangKeyword
Disallow */pf/getProductSelectedVariant
Disallow */pf/productvariants
Disallow *page%3D0
Disallow *page%3D-1
Disallow *pageSize%3D*%26
Disallow */perse/getNoResultsPage
Disallow */prefillAddToCart
Disallow */priceToolTip/*
Disallow */product360view/*
Disallow */productBranchStock
Disallow */productBranchStock/*
Disallow */punchoutCxmlCart/punchout
Disallow */quickorder
Disallow */recommendedCategories.ajax
Disallow */register
Disallow */register/*
Disallow */removeChangeRegionPopup
Disallow */removeGDPRLegalNotification
Disallow */removePCIComplianceNotification
Disallow */replacementProduct
Disallow */sales/*
Disallow */updatelanguage
Disallow */updateRegion
Disallow */validateUpdateRegion
Disallow */view/TabbedNavigationComponentController/updateTabIndex
Disallow */p/*/zoomImages
Disallow *%7B0%7D
Disallow *%7B0%7D
Disallow */cms/getPDPContent.json?productCode=*
Allow *%3A*%3A*/p/*
Disallow /search*.com
Disallow /search*.net
Disallow */p/NCP*
Disallow */MarketingKampagnen/*
Disallow */Personal-Catalog/*
Disallow */bonusaktion*
Disallow *%3Arelevance%3A*%3A*
Disallow *%3Arelevance%3A*%3A*
Disallow *%3A*%3A*
Disallow *%3A*%3A*
Disallow */c/*__*
Disallow */c/*/*productbranchstock*
Disallow *page%3D-1*
Disallow */c/*?q=%3Arelevance&text=&filter=&aliasname=&partialSearch=
Disallow */Hersteller/*/c/*q%3D%3Arelevance%3Acategory%3AALLITEMS*
Disallow */Kategorien/*/c/*relevance%3Acategory*
Disallow
Disallow */Hersteller/*/c/*relevance%3Abrand*
Allow */Kategorien/*/c/*q%3D%3Arelevance%3Abrand%3A*
Disallow */Kategorien/*/c/*q%3D%3Arelevance%3Abrand%3A*page%3D-1*
Allow */Hersteller/*/c/*q%3D%3Arelevance%3Acategory%3A*
Disallow */Hersteller/*/c/*q%3D%3Arelevance%3Acategory%3A*%3A
Disallow */Hersteller/*/c/*q%3D%3Arelevance%3Acategory%3A*page%3D-1*

bytespider

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

cazoodlebot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot/1.0

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

aspiegelbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

gptbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.rexel.de/sitemap.xml
Back to top

Comments

  • cms related
  • testing
  • allow odd PDP combinations
  • spam pattern
  • site specific blocks
  • old pagination facets - tests
  • PERSE
  • Limit PLP Facet Crawling
  • Disallow facets but those allowed elsewhere
  • disallow other
  • disallow facet removal
  • disallow category facet on category pages
  • Disallow brand facet on brand pages
  • disallow multi facet selections on category and brand
  • Allow / Disallow Brand facets for category pages
  • Allow / Disallow Category Facets for Brand pages
  • v20231211
Back to top

Warnings

  • 1 invalid line.
Back to top