Log In Sign Up

shop.topsmarkets.com
robots.txt

Robots Exclusion Standard data for shop.topsmarkets.com

Resource Scan

Scan Details

Site Domain shop.topsmarkets.com
Base Domain topsmarkets.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2026-01-01T07:26:27+00:00
Next Scan 2026-01-08T07:26:27+00:00

Last Successful Scan

Scanned2025-12-16T16:22:15+00:00
URL https://shop.topsmarkets.com/robots.txt
Domain IPs 13.33.88.12, 13.33.88.121, 13.33.88.21, 13.33.88.71
Response IP 13.33.88.71
Found Yes
Hash d08803d359bf67e44f9a248641831016b9cf4f4fbd893ae864ab56134498df1f
SimHash 35b6731195f6

Groups

googlebot

Rule Path
Allow /
Disallow /accounts/unlock
Disallow /api/
Disallow /ex/
Disallow /orders/
Disallow /store/orders/
Disallow /redeem_coupon_token
Disallow /store/order_changes/
Disallow /store/*/browse_departments*
Disallow /store/*/ng/l/*
Disallow /store/referrals*
Disallow /store/item*
Disallow /store/pickup/status/*
Disallow /store/*/buy_it_again*
Disallow /store/*/l/*
Disallow /widgets/*
Disallow /client-side-redirect/*
Disallow /track/*
Disallow /affiliate-program*
Disallow /family/invite/*
Disallow /family/accept/*
Disallow /rate/orders/*
Disallow /store_manager_admin
Disallow /rest/v8/
Disallow /login?next=
Disallow /*?*interaction_id=
Disallow /*?*item_detail_product_id=
Disallow /store/*/collections/*?*sisid=
Disallow /store/*/*?*xrs_id=

googlebot-image

Rule Path
Allow /
Disallow /accounts/unlock
Disallow /api/
Disallow /ex/
Disallow /orders/
Disallow /store/orders/
Disallow /redeem_coupon_token
Disallow /store/order_changes/
Disallow /store/*/browse_departments*
Disallow /store/*/ng/l/*
Disallow /store/referrals*
Disallow /store/item*
Disallow /store/pickup/status/*
Disallow /store/*/buy_it_again*
Disallow /store/*/l/*
Disallow /widgets/*
Disallow /client-side-redirect/*
Disallow /track/*
Disallow /affiliate-program*
Disallow /family/invite/*
Disallow /family/accept/*
Disallow /rate/orders/*
Disallow /store_manager_admin
Disallow /rest/v8/
Disallow /login?next=
Disallow /*?*interaction_id=
Disallow /*?*item_detail_product_id=
Disallow /store/*/collections/*?*sisid=
Disallow /store/*/*?*xrs_id=

googlebot-mobile

Rule Path
Allow /
Disallow /accounts/unlock
Disallow /api/
Disallow /ex/
Disallow /orders/
Disallow /store/orders/
Disallow /redeem_coupon_token
Disallow /store/order_changes/
Disallow /store/*/browse_departments*
Disallow /store/*/ng/l/*
Disallow /store/referrals*
Disallow /store/item*
Disallow /store/pickup/status/*
Disallow /store/*/buy_it_again*
Disallow /store/*/l/*
Disallow /widgets/*
Disallow /client-side-redirect/*
Disallow /track/*
Disallow /affiliate-program*
Disallow /family/invite/*
Disallow /family/accept/*
Disallow /rate/orders/*
Disallow /store_manager_admin
Disallow /rest/v8/
Disallow /login?next=
Disallow /*?*interaction_id=
Disallow /*?*item_detail_product_id=
Disallow /store/*/collections/*?*sisid=
Disallow /store/*/*?*xrs_id=

storebot-google

Rule Path
Allow /
Disallow /accounts/unlock
Disallow /api/
Disallow /ex/
Disallow /orders/
Disallow /store/orders/
Disallow /redeem_coupon_token
Disallow /store/order_changes/
Disallow /store/*/browse_departments*
Disallow /store/*/ng/l/*
Disallow /store/referrals*
Disallow /store/item*
Disallow /store/pickup/status/*
Disallow /store/*/buy_it_again*
Disallow /store/*/l/*
Disallow /widgets/*
Disallow /client-side-redirect/*
Disallow /track/*
Disallow /affiliate-program*
Disallow /family/invite/*
Disallow /family/accept/*
Disallow /rate/orders/*
Disallow /store_manager_admin
Disallow /rest/v8/
Disallow /login?next=
Disallow /*?*interaction_id=
Disallow /*?*item_detail_product_id=
Disallow /store/*/collections/*?*sisid=
Disallow /store/*/*?*xrs_id=

adsbot-google-mobile

Rule Path
Allow /
Disallow /accounts/unlock
Disallow /api/
Disallow /ex/
Disallow /orders/
Disallow /store/orders/
Disallow /redeem_coupon_token
Disallow /store/order_changes/
Disallow /store/*/browse_departments*
Disallow /store/*/ng/l/*
Disallow /store/referrals*
Disallow /store/item*
Disallow /store/pickup/status/*
Disallow /store/*/buy_it_again*
Disallow /store/*/l/*
Disallow /widgets/*
Disallow /client-side-redirect/*
Disallow /track/*
Disallow /affiliate-program*
Disallow /family/invite/*
Disallow /family/accept/*
Disallow /rate/orders/*
Disallow /store_manager_admin
Disallow /rest/v8/
Disallow /login?next=
Disallow /*?*interaction_id=
Disallow /*?*item_detail_product_id=
Disallow /store/*/collections/*?*sisid=
Disallow /store/*/*?*xrs_id=

adsbot-google

Rule Path
Allow /
Disallow /accounts/unlock
Disallow /api/
Disallow /ex/
Disallow /orders/
Disallow /store/orders/
Disallow /redeem_coupon_token
Disallow /store/order_changes/
Disallow /store/*/browse_departments*
Disallow /store/*/ng/l/*
Disallow /store/referrals*
Disallow /store/item*
Disallow /store/pickup/status/*
Disallow /store/*/buy_it_again*
Disallow /store/*/l/*
Disallow /widgets/*
Disallow /client-side-redirect/*
Disallow /track/*
Disallow /affiliate-program*
Disallow /family/invite/*
Disallow /family/accept/*
Disallow /rate/orders/*
Disallow /store_manager_admin
Disallow /rest/v8/
Disallow /login?next=
Disallow /*?*interaction_id=
Disallow /*?*item_detail_product_id=
Disallow /store/*/collections/*?*sisid=
Disallow /store/*/*?*xrs_id=

bingbot

Rule Path
Allow /
Disallow /accounts/unlock
Disallow /api/
Disallow /ex/
Disallow /orders/
Disallow /store/orders/
Disallow /redeem_coupon_token
Disallow /store/order_changes/
Disallow /store/*/browse_departments*
Disallow /store/*/ng/l/*
Disallow /store/referrals*
Disallow /store/item*
Disallow /store/pickup/status/*
Disallow /store/*/buy_it_again*
Disallow /store/*/l/*
Disallow /widgets/*
Disallow /client-side-redirect/*
Disallow /track/*
Disallow /affiliate-program*
Disallow /family/invite/*
Disallow /family/accept/*
Disallow /rate/orders/*
Disallow /store_manager_admin
Disallow /rest/v8/
Disallow /login?next=
Disallow /*?*interaction_id=
Disallow /*?*item_detail_product_id=
Disallow /store/*/collections/*?*sisid=
Disallow /store/*/*?*xrs_id=%C3%82

slurp

Rule Path
Allow /
Disallow /accounts/unlock
Disallow /api/
Disallow /ex/
Disallow /orders/
Disallow /store/orders/
Disallow /redeem_coupon_token
Disallow /store/order_changes/
Disallow /store/*/browse_departments*
Disallow /store/*/ng/l/*
Disallow /store/referrals*
Disallow /store/item*
Disallow /store/pickup/status/*
Disallow /store/*/buy_it_again*
Disallow /store/*/l/*
Disallow /widgets/*
Disallow /client-side-redirect/*
Disallow /track/*
Disallow /affiliate-program*
Disallow /family/invite/*
Disallow /family/accept/*
Disallow /rate/orders/*
Disallow /store_manager_admin
Disallow /rest/v8/
Disallow /login?next=
Disallow /*?*interaction_id=
Disallow /*?*item_detail_product_id=
Disallow /store/*/collections/*?*sisid=
Disallow /store/*/*?*xrs_id=

duckduckgo

Rule Path
Allow /
Disallow /accounts/unlock
Disallow /api/
Disallow /ex/
Disallow /orders/
Disallow /store/orders/
Disallow /redeem_coupon_token
Disallow /store/order_changes/
Disallow /store/*/browse_departments*
Disallow /store/*/ng/l/*
Disallow /store/referrals*
Disallow /store/item*
Disallow /store/pickup/status/*
Disallow /store/*/buy_it_again*
Disallow /store/*/l/*
Disallow /widgets/*
Disallow /client-side-redirect/*
Disallow /track/*
Disallow /affiliate-program*
Disallow /family/invite/*
Disallow /family/accept/*
Disallow /rate/orders/*
Disallow /store_manager_admin
Disallow /rest/v8/
Disallow /login?next=
Disallow /*?*interaction_id=
Disallow /*?*item_detail_product_id=
Disallow /store/*/collections/*?*sisid=
Disallow /store/*/*?*xrs_id=

oai-searchbot

Rule Path
Allow /
Disallow /accounts/unlock
Disallow /api/
Disallow /ex/
Disallow /orders/
Disallow /store/orders/
Disallow /redeem_coupon_token
Disallow /store/order_changes/
Disallow /store/*/browse_departments*
Disallow /store/*/ng/l/*
Disallow /store/referrals*
Disallow /store/item*
Disallow /store/pickup/status/*
Disallow /store/*/buy_it_again*
Disallow /store/*/l/*
Disallow /widgets/*
Disallow /client-side-redirect/*
Disallow /track/*
Disallow /affiliate-program*
Disallow /family/invite/*
Disallow /family/accept/*
Disallow /rate/orders/*
Disallow /store_manager_admin
Disallow /rest/v8/
Disallow /login?next=
Disallow /*?*interaction_id=
Disallow /*?*item_detail_product_id=
Disallow /store/*/collections/*?*sisid=
Disallow /store/*/*?*xrs_id=

perplexitybot

Rule Path
Allow /
Disallow /accounts/unlock
Disallow /api/
Disallow /ex/
Disallow /orders/
Disallow /store/orders/
Disallow /redeem_coupon_token
Disallow /store/order_changes/
Disallow /store/*/browse_departments*
Disallow /store/*/ng/l/*
Disallow /store/referrals*
Disallow /store/item*
Disallow /store/pickup/status/*
Disallow /store/*/buy_it_again*
Disallow /store/*/l/*
Disallow /widgets/*
Disallow /client-side-redirect/*
Disallow /track/*
Disallow /affiliate-program*
Disallow /family/invite/*
Disallow /family/accept/*
Disallow /rate/orders/*
Disallow /store_manager_admin
Disallow /rest/v8/
Disallow /login?next=
Disallow /*?*interaction_id=
Disallow /*?*item_detail_product_id=
Disallow /store/*/collections/*?*sisid=
Disallow /store/*/*?*xrs_id=

applebot

Rule Path
Allow /
Disallow /accounts/unlock
Disallow /api/
Disallow /ex/
Disallow /orders/
Disallow /store/orders/
Disallow /redeem_coupon_token
Disallow /store/order_changes/
Disallow /store/*/browse_departments*
Disallow /store/*/ng/l/*
Disallow /store/referrals*
Disallow /store/item*
Disallow /store/pickup/status/*
Disallow /store/*/buy_it_again*
Disallow /store/*/l/*
Disallow /widgets/*
Disallow /client-side-redirect/*
Disallow /track/*
Disallow /affiliate-program*
Disallow /family/invite/*
Disallow /family/accept/*
Disallow /rate/orders/*
Disallow /store_manager_admin
Disallow /rest/v8/
Disallow /login?next=
Disallow /*?*interaction_id=
Disallow /*?*item_detail_product_id=
Disallow /store/*/collections/*?*sisid=
Disallow /store/*/*?*xrs_id=

linkedinbot

Rule Path
Allow /
Disallow /accounts/unlock
Disallow /api/
Disallow /ex/
Disallow /orders/
Disallow /store/orders/
Disallow /redeem_coupon_token
Disallow /store/order_changes/
Disallow /store/*/browse_departments*
Disallow /store/*/ng/l/*
Disallow /store/referrals*
Disallow /store/item*
Disallow /store/pickup/status/*
Disallow /store/*/buy_it_again*
Disallow /store/*/l/*
Disallow /widgets/*
Disallow /client-side-redirect/*
Disallow /track/*
Disallow /affiliate-program*
Disallow /family/invite/*
Disallow /family/accept/*
Disallow /rate/orders/*
Disallow /store_manager_admin
Disallow /rest/v8/
Disallow /login?next=
Disallow /*?*interaction_id=
Disallow /*?*item_detail_product_id=
Disallow /store/*/collections/*?*sisid=
Disallow /store/*/*?*xrs_id=

twitterbot

Rule Path
Allow /
Disallow /accounts/unlock
Disallow /api/
Disallow /ex/
Disallow /orders/
Disallow /store/orders/
Disallow /redeem_coupon_token
Disallow /store/order_changes/
Disallow /store/*/browse_departments*
Disallow /store/*/ng/l/*
Disallow /store/referrals*
Disallow /store/item*
Disallow /store/pickup/status/*
Disallow /store/*/buy_it_again*
Disallow /store/*/l/*
Disallow /widgets/*
Disallow /client-side-redirect/*
Disallow /track/*
Disallow /affiliate-program*
Disallow /family/invite/*
Disallow /family/accept/*
Disallow /rate/orders/*
Disallow /store_manager_admin
Disallow /rest/v8/
Disallow /login?next=
Disallow /*?*interaction_id=
Disallow /*?*item_detail_product_id=
Disallow /store/*/collections/*?*sisid=
Disallow /store/*/*?*xrs_id=

facebookexternalhit

Rule Path
Disallow /assets

facebot

Rule Path
Disallow /assets

slackbot-linkexpanding

Rule Path
Allow /
Disallow /accounts/unlock
Disallow /api/
Disallow /ex/
Disallow /orders/
Disallow /store/orders/
Disallow /redeem_coupon_token
Disallow /store/order_changes/
Disallow /store/*/browse_departments*
Disallow /store/*/ng/l/*
Disallow /store/referrals*
Disallow /store/item*
Disallow /store/pickup/status/*
Disallow /store/*/buy_it_again*
Disallow /store/*/l/*
Disallow /widgets/*
Disallow /client-side-redirect/*
Disallow /track/*
Disallow /affiliate-program*
Disallow /family/invite/*
Disallow /family/accept/*
Disallow /rate/orders/*
Disallow /store_manager_admin
Disallow /rest/v8/
Disallow /login?next=
Disallow /*?*interaction_id=
Disallow /*?*item_detail_product_id=
Disallow /store/*/collections/*?*sisid=
Disallow /store/*/*?*xrs_id=

chatgpt-user

Rule Path
Allow /store/partner_recipes

twitterbot

Rule Path
Allow /store/list/*

*

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://shop.topsmarkets.com/sitemaps/storefront_pro/shop_topsmarkets_com/sitemap.xml
Back to top

Comments

  • Any bot scraping or crawling this site must abide by
  • Instacart's Terms of Service https://www.instacart.com/terms
  • Last Updated 3-10-2025
  • Rules for Google search
  • Rules for Bing Search and powered by Bing search
  • Rules for Yahoo! Search
  • Rules for DuckDuckGo Search
  • Rules for Additional Search Engines
  • Rules for Social Sharing and Link Previews
  • Additional Rules from legacy file
  • All other scrapers compliance directive
  • If you're not a bot, we're hiring: https://instacart.careers/current-openings/
  • Sitemap
Back to top