m.aboutyou.se
robots.txt

Robots Exclusion Standard data for m.aboutyou.se

Resource Scan

Scan Details

Site Domain m.aboutyou.se
Base Domain aboutyou.se
Scan Status Ok
Last Scan2024-05-17T10:19:46+00:00
Next Scan 2024-05-31T10:19:46+00:00

Last Scan

Scanned2024-05-17T10:19:46+00:00
URL https://m.aboutyou.se/robots.txt
Redirect https://www.aboutyou.se/robots.txt
Redirect Domain www.aboutyou.se
Redirect Base aboutyou.se
Domain IPs 104.18.6.187, 104.18.7.187, 2606:4700::6812:6bb, 2606:4700::6812:7bb
Redirect IPs 104.18.6.187, 104.18.7.187, 2606:4700::6812:6bb, 2606:4700::6812:7bb
Response IP 104.18.7.187
Found Yes
Hash f05cef5440b86973a10ab9c6800fb8a08e13eb510f4a9356cf08c6453f2b35d7
SimHash 237447903c11

Groups

*

Rule Path Comment
Allow * -
Disallow /*/similar -
Disallow /a/ -
Disallow /cs/ -
Disallow /cw -
Disallow /redirect-url -
Disallow /kasse de
Disallow /kasa pl
Disallow /kassa ee,nl
Disallow /finaliser-la-commande fr
Disallow /pokladna cz, sk
Disallow /kassza hu
Disallow /casa ro
Disallow /blagajna si #hr
Disallow /apmoketi lt
Disallow /kase lv
Disallow /checkout bg, en
Disallow /caja es
Disallow /cassa it
Disallow /kassan se
Disallow /%CF%84%CE%B1%CE%BC%CE%B5%CE%AF%CE%BF gr
Disallow /caixa pt
Disallow /search mobile, bg, en, fi, dk, se
Disallow /suche de
Disallow /zoeken nl
Disallow /recherche fr
Disallow /szukaj pl
Disallow /hledani cz
Disallow /kereses hu
Disallow /cautare ro
Disallow /hladanie sk
Disallow /iskanje si
Disallow /ieskoti lt
Disallow /meklet lv
Disallow /otsi ee
Disallow /pretrazi hr
Disallow /busqueda es
Disallow /ricerca it
Disallow /procurar pt
Disallow /sok no
Disallow /*?*price_max= -
Disallow /*?*maxPrice= -
Disallow /*?*price_min= -
Disallow /*?*minPrice= -
Disallow /*?*quantityPerPack= -
Disallow /*?*sale= -
Disallow /*?*quickSale= -
Disallow /*?*genderage= -
Disallow /*?*filter= -
Disallow /*?*isNew= -
Disallow /*?*isNewQuick= -
Disallow /*?*styleGroup= -
Disallow /*?*showFilterMenu= -
Disallow /*?*discount= -
Disallow /*?*prices= -
Disallow /*?*hasCampaignReduction= -
Allow /*?*page=1& -
Allow /*?*page=2& -
Allow /*?*page=3& -
Allow /*?*page=1$ -
Allow /*?*page=2$ -
Allow /*?*page=3$ -
Disallow /*?*page= -
Disallow /api/health -
Disallow /__/logs -
Disallow /b/discover/ -

ahrefsbot
semrushbot
blexbot
dotbot
mj12bot
barkrowler
sistrix
optimizer

Rule Path
Disallow /

Comments

  • block "similar brands" (linked from brand-profiles)
  • block login-pages
  • block app-pages
  • block "checkout" (linked from source code of warenkorb)
  • block "search"
  • block irrelevant filters (forever)
  • limit pagination
  • block API
  • block discover
  • sitemap se
  • block crawler

Warnings

  • 1 invalid line.
  • `https` is not a known field.