medino.com
robots.txt

Groups

*

Other Records

Comments

• Don't allow crawlers to check out or add to basket.
• This wastes crawl budget, and we don't want to show this in search results anyway.
• Blocking these pages here will cause issues in search console, as it will still try to
• index them, but will be blocked by this file. Without blocking them we get other alerts
• in the search console anyway, so it's a catch-22. We opted to block them here since we don't
• want to index them anyway.
• It's not possible to block the checkout with the `noindex` html tag,
• as it'll 302 to basket if your basket is empty. And if you redirect there is no HTML to add the tag to.
• Likewise, the add to basket endpoint does not return HTML, so we have no option to add a `no index` tag.