Log In Sign Up

rexel.de
robots.txt

Robots Exclusion Standard data for rexel.de

Resource Scan

Scan Details

Site Domain rexel.de
Base Domain rexel.de
Scan Status Ok
Last Scan2024-06-16T04:11:25+00:00
Next Scan 2024-06-30T04:11:25+00:00

Last Scan

Scanned2024-06-16T04:11:25+00:00
URL https://rexel.de/robots.txt
Redirect https://www.rexel.de/robots.txt
Redirect Domain www.rexel.de
Redirect Base rexel.de
Domain IPs 13.107.246.73
Redirect IPs 13.107.246.59, 2620:1ec:bdf::59
Response IP 13.107.246.59
Found Yes
Hash d457a2b0f42d40bb328b24fbe78c1979fce232ba098da9f43206106692f52daa
SimHash c1035a576a10

Groups

*

Rule Path
Disallow */cart
Disallow */cart/*
Disallow *_%3D*
Disallow */*/c/brands
Disallow *//c/
Disallow */_s/saveFulfillmentJson
Disallow */addBulkProductItemsToCart
Disallow */addBulkProductItemsToCartReplacementModel
Disallow */addCustomerUserNboProductTocart
Disallow */addObseleteReplacementProductItemsToCart
Disallow */addProductsToCartFormConfigurator
Disallow */addToCartFromUrl
Disallow */addtoJoblist/*
Disallow */c/.seo-collapse
Disallow */**/c/browseCategoryAndBrandForMobile
Disallow */checkout
Disallow */checkout/*
Disallow */closeGlobalPromotion
Disallow */closeNotificationIcon
Disallow *collapseTwo
Disallow */combined.css*
Disallow */combined.js*
Disallow */compare/
Disallow */cookieCosent
Disallow */cookieCosentToggler
Disallow */deliveryDateSimulatorData.json
Disallow */enableCookieConsent
Disallow */erp/*
Disallow */fast-register
Disallow */findDefaultStore
Disallow */findNearestStoreForAnonymous
Disallow */footer/globalSites
Disallow */GDPRCookieContainerComponentController/*
Disallow */getBranchList*
Disallow */getHtmlFormContent
Disallow */getInspirationCompanyUrl
Disallow */getJWTTokenAndUserDetails
Disallow */getLimitedDistributionPickupZip
Disallow */getNotificationMsgvaluesForLD
Disallow */isQuantityVisibleForProducts
Disallow */oauth/getOauthToken
Disallow */getHtmlFormContent
Disallow */getPagablePickupBranchesForAnonymous.json
Disallow */getRegionList
Disallow */getReplacementForObsoleteproduct
Disallow */getTieredPricingMap
Disallow */getJWTTokenAndUserDetails
Disallow */isQuantitySectionVisible
Disallow */isQuantityVisibleForProducts
Disallow */login
Disallow */login/*
Disallow */logout
Disallow */mergeCart/*
Disallow */mobilelogin*
Disallow */mobileprivacy
Disallow */mobileRegistration*
Disallow */mockMedia/*
Disallow */mostordered/getcategories
Disallow */my-account
Disallow */my-account/*
Disallow */myaccountpopup
Disallow */my-company/*
Disallow */mysettings
Disallow */newQuickorder/*
Disallow */notification/*
Disallow */oauth/getOauthToken
Disallow */obseleteProductAddToCart
Disallow */order-return/*
Disallow */order-new-return/*
Disallow */p/getEmailPDPpopup
Disallow */p/getTieredPricing
Disallow */p/slangKeyword
Disallow */pf/getProductSelectedVariant
Disallow */pf/productvariants
Disallow *page%3D0
Disallow *page%3D-1
Disallow *pageSize%3D*%26
Disallow */perse/getNoResultsPage
Disallow */prefillAddToCart
Disallow */priceToolTip/*
Disallow */product360view/*
Disallow */productBranchStock
Disallow */productBranchStock/*
Disallow */punchoutCxmlCart/punchout
Disallow */quickorder
Disallow */recommendedCategories.ajax
Disallow */register
Disallow */register/*
Disallow */removeChangeRegionPopup
Disallow */removeGDPRLegalNotification
Disallow */removePCIComplianceNotification
Disallow */replacementProduct
Disallow */sales/*
Disallow */updatelanguage
Disallow */updateRegion
Disallow */validateUpdateRegion
Disallow */view/TabbedNavigationComponentController/updateTabIndex
Disallow */p/*/zoomImages
Disallow *%7B0%7D
Disallow *%7B0%7D
Disallow */cms/getPDPContent.json?productCode=*
Allow *%3A*%3A*/p/*
Disallow /search*.com
Disallow /search*.net
Disallow */p/NCP*
Disallow */MarketingKampagnen/*
Disallow */Personal-Catalog/*
Disallow */bonusaktion*
Disallow *%3Arelevance%3A*%3A*
Disallow *%3Arelevance%3A*%3A*
Disallow *%3A*%3A*
Disallow *%3A*%3A*
Disallow */c/*__*
Disallow */c/*/*productbranchstock*
Disallow *page%3D-1*
Disallow */c/*?q=%3Arelevance&text=&filter=&aliasname=&partialSearch=
Disallow */Hersteller/*/c/*q%3D%3Arelevance%3Acategory%3AALLITEMS*
Disallow */Kategorien/*/c/*relevance%3Acategory*
Disallow
Disallow */Hersteller/*/c/*relevance%3Abrand*
Allow */Kategorien/*/c/*q%3D%3Arelevance%3Abrand%3A*
Disallow */Kategorien/*/c/*q%3D%3Arelevance%3Abrand%3A*page%3D-1*
Allow */Hersteller/*/c/*q%3D%3Arelevance%3Acategory%3A*
Disallow */Hersteller/*/c/*q%3D%3Arelevance%3Acategory%3A*%3A
Disallow */Hersteller/*/c/*q%3D%3Arelevance%3Acategory%3A*page%3D-1*

bytespider

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

cazoodlebot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot/1.0

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

aspiegelbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

gptbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.rexel.de/sitemap.xml
Back to top

Comments

  • cms related
  • testing
  • allow odd PDP combinations
  • spam pattern
  • site specific blocks
  • old pagination facets - tests
  • PERSE
  • Limit PLP Facet Crawling
  • Disallow facets but those allowed elsewhere
  • disallow other
  • disallow facet removal
  • disallow category facet on category pages
  • Disallow brand facet on brand pages
  • disallow multi facet selections on category and brand
  • Allow / Disallow Brand facets for category pages
  • Allow / Disallow Category Facets for Brand pages
  • v20231211
Back to top

Warnings

  • 1 invalid line.
Back to top