m.aboutyou.com
robots.txt

Robots Exclusion Standard data for m.aboutyou.com

Resource Scan

Scan Details

Site Domain m.aboutyou.com
Base Domain aboutyou.com
Scan Status Ok
Last Scan2024-06-12T16:36:57+00:00
Next Scan 2024-06-26T16:36:57+00:00

Last Scan

Scanned2024-06-12T16:36:57+00:00
URL https://m.aboutyou.com/robots.txt
Redirect https://www.aboutyou.com/robots.txt
Redirect Domain www.aboutyou.com
Redirect Base aboutyou.com
Domain IPs 104.18.8.55, 104.18.9.55, 2606:4700::6812:837, 2606:4700::6812:937
Redirect IPs 104.18.8.55, 104.18.9.55, 2606:4700::6812:837, 2606:4700::6812:937
Response IP 104.18.9.55
Found Yes
Hash e9b7eeb9b564329c7983ef86d615e841f3f32372da716b35bf42a42429e8aa12
SimHash 23744f90bc11

Groups

*

Rule Path Comment
Allow * -
Disallow /*/similar -
Disallow /a/ -
Disallow /cs/ -
Disallow /cw -
Disallow /redirect-url -
Disallow /kasse de
Disallow /kasa pl
Disallow /kassa ee,nl
Disallow /finaliser-la-commande fr
Disallow /pokladna cz, sk
Disallow /kassza hu
Disallow /casa ro
Disallow /blagajna si #hr
Disallow /apmoketi lt
Disallow /kase lv
Disallow /checkout bg, en
Disallow /caja es
Disallow /cassa it
Disallow /kassan se
Disallow /%CF%84%CE%B1%CE%BC%CE%B5%CE%AF%CE%BF gr
Disallow /caixa pt
Disallow /search mobile, bg, en, fi, dk, se
Disallow /suche de
Disallow /zoeken nl
Disallow /recherche fr
Disallow /szukaj pl
Disallow /hledani cz
Disallow /kereses hu
Disallow /cautare ro
Disallow /hladanie sk
Disallow /iskanje si
Disallow /ieskoti lt
Disallow /meklet lv
Disallow /otsi ee
Disallow /pretrazi hr
Disallow /busqueda es
Disallow /ricerca it
Disallow /procurar pt
Disallow /sok no
Disallow /*?*price_max= -
Disallow /*?*maxPrice= -
Disallow /*?*price_min= -
Disallow /*?*minPrice= -
Disallow /*?*quantityPerPack= -
Disallow /*?*sale= -
Disallow /*?*quickSale= -
Disallow /*?*genderage= -
Disallow /*?*filter= -
Disallow /*?*isNew= -
Disallow /*?*isNewQuick= -
Disallow /*?*styleGroup= -
Disallow /*?*showFilterMenu= -
Disallow /*?*discount= -
Disallow /*?*prices= -
Disallow /*?*hasCampaignReduction= -
Allow /*?*page=1& -
Allow /*?*page=2& -
Allow /*?*page=3& -
Allow /*?*page=1$ -
Allow /*?*page=2$ -
Allow /*?*page=3$ -
Disallow /*?*page= -
Disallow /api/health -
Disallow /__/logs -
Disallow /b/discover/ -

ahrefsbot
semrushbot
blexbot
dotbot
mj12bot
barkrowler
sistrix
optimizer

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.aboutyou.com/sitemap/zz/sitemap.xml

Comments

  • block "similar brands" (linked from brand-profiles)
  • block login-pages
  • block app-pages
  • block "checkout" (linked from source code of warenkorb)
  • block "search"
  • block irrelevant filters (forever)
  • limit pagination
  • block API
  • block discover
  • sitemap en-DE
  • block crawler