m-en.aboutyou.de
robots.txt

Robots Exclusion Standard data for m-en.aboutyou.de

Resource Scan

Scan Details

Site Domain m-en.aboutyou.de
Base Domain aboutyou.de
Scan Status Ok
Last Scan2024-09-25T20:30:04+00:00
Next Scan 2024-10-02T20:30:04+00:00

Last Scan

Scanned2024-09-25T20:30:04+00:00
URL https://m-en.aboutyou.de/robots.txt
Redirect https://en.aboutyou.de/robots.txt
Redirect Domain en.aboutyou.de
Redirect Base aboutyou.de
Domain IPs 104.18.18.83, 104.18.19.83, 2606:4700::6812:1253, 2606:4700::6812:1353
Redirect IPs 104.18.18.83, 104.18.19.83, 2606:4700::6812:1253, 2606:4700::6812:1353
Response IP 104.18.18.83
Found Yes
Hash 14099d3c8282183063c56fae278f1e1ec76ecf8ef910bdc5c44357ba9d3fef8d
SimHash 2f744b10bc81

Groups

*

Rule Path Comment
Allow * -
Disallow /*/similar -
Disallow /a/ -
Disallow /cs -
Disallow /cw -
Disallow /redirect-url -
Disallow /kasse de
Disallow /kasa pl
Disallow /kassa ee,nl
Disallow /finaliser-la-commande fr
Disallow /pokladna cz, sk
Disallow /kassza hu
Disallow /casa ro
Disallow /blagajna si #hr
Disallow /apmoketi lt
Disallow /kase lv
Disallow /checkout bg, en
Disallow /caja es
Disallow /cassa it
Disallow /kassan se
Disallow /%CF%84%CE%B1%CE%BC%CE%B5%CE%AF%CE%BF gr
Disallow /caixa pt
Disallow /search mobile, bg, en, fi, dk, se
Disallow /suche de
Disallow /zoeken nl
Disallow /recherche fr
Disallow /szukaj pl
Disallow /hledani cz
Disallow /kereses hu
Disallow /cautare ro
Disallow /hladanie sk
Disallow /iskanje si
Disallow /ieskoti lt
Disallow /meklet lv
Disallow /otsi ee
Disallow /pretrazi hr
Disallow /busqueda es
Disallow /ricerca it
Disallow /procurar pt
Disallow /sok no
Disallow /*?*price_max= -
Disallow /*?*maxPrice= -
Disallow /*?*price_min= -
Disallow /*?*minPrice= -
Disallow /*?*quantityPerPack= -
Disallow /*?*sale= -
Disallow /*?*quickSale= -
Disallow /*?*genderage= -
Disallow /*?*filter= -
Disallow /*?*isNew= -
Disallow /*?*isNewQuick= -
Disallow /*?*styleGroup= -
Disallow /*?*showFilterMenu= -
Disallow /*?*discount= -
Disallow /*?*prices= -
Disallow /*?*hasCampaignReduction= -
Disallow /*?*firstProductId= -
Allow /p/*?*firstProductId= -
Disallow /*?*cpn= -
Allow /p/*?*cpn= -
Allow /*?*page=1& -
Allow /*?*page=2& -
Allow /*?*page=3& -
Allow /*?*page=1$ -
Allow /*?*page=2$ -
Allow /*?*page=3$ -
Disallow /*?*page= -
Disallow /api/health -
Disallow /__/logs -
Disallow /__/legacy_state?url=%2Fp%2F -
Disallow /__/legacy_state?url=%2Fc%2F -
Disallow /__/legacy_state?url=%2Fb%2F -
Disallow /b/discover/ -

adsbot-google

Rule Path
Disallow /__/logs

ahrefsbot
semrushbot
blexbot
dotbot
mj12bot
barkrowler
sistrix
optimizer

Rule Path
Disallow /

Other Records

Field Value
sitemap https://en.aboutyou.de/sitemap/deen/sitemap.xml.gz

Comments

  • block "similar brands" (linked from brand-profiles)
  • block login-pages
  • block app-pages
  • block "checkout" (linked from source code of warenkorb)
  • block "search"
  • block irrelevant filters (forever)
  • limit pagination
  • block API
  • block discover
  • sitemap en-DE
  • block crawler