inst.cr
robots.txt
Robots Exclusion Standard data for inst.cr
Resource Scan
Scan Details
| Site Domain | inst.cr |
| Base Domain | inst.cr |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2026-01-09T03:39:14+00:00 |
| Next Scan | 2026-01-23T03:39:14+00:00 |
Last Successful Scan
| Scanned | 2025-12-16T17:10:03+00:00 |
| URL | https://inst.cr/robots.txt |
| Redirect | https://www.instacart.com/robots.txt |
| Redirect Domain | www.instacart.com |
| Redirect Base | instacart.com |
| Domain IPs | 3.171.198.22, 3.171.198.33, 3.171.198.58, 3.171.198.91 |
| Redirect IPs | 13.33.45.116, 13.33.45.128, 13.33.45.61, 13.33.45.74 |
| Response IP | 13.35.37.63 |
| Found | Yes |
| Hash | 5d8e03eec515026f82bc76a94cd6164068bf7e3ca8414e2ae98d1cf19644b9c2 |
| SimHash | 35bef313d5f6 |
Groups
googlebot
| Rule | Path |
|---|---|
| Allow | / |
| Allow | /store/brands/l/pads-tampons-liners-wipes |
| Disallow | /accounts/unlock |
| Disallow | /api/ |
| Disallow | /ex/ |
| Disallow | /orders/ |
| Disallow | /store/orders/ |
| Disallow | /redeem_coupon_token |
| Disallow | /store/order_changes/ |
| Disallow | /store/*/browse_departments* |
| Disallow | /store/*/ng/l/* |
| Disallow | /store/referrals* |
| Disallow | /store/item* |
| Disallow | /store/pickup/status/* |
| Disallow | /store/*/buy_it_again* |
| Disallow | /store/*/l/* |
| Disallow | /widgets/* |
| Disallow | /client-side-redirect/* |
| Disallow | /track/* |
| Disallow | /affiliate-program* |
| Disallow | /family/invite/* |
| Disallow | /family/accept/* |
| Disallow | /rate/orders/* |
| Disallow | /store_manager_admin |
| Disallow | /rest/v8/ |
| Disallow | /login?next= |
| Disallow | /*?*interaction_id= |
| Disallow | /*?*item_detail_product_id= |
| Disallow | /store/*/collections/*?*sisid= |
| Disallow | /store/*/*?*xrs_id= |
googlebot-image
| Rule | Path |
|---|---|
| Allow | / |
| Allow | /store/brands/l/pads-tampons-liners-wipes |
| Disallow | /accounts/unlock |
| Disallow | /api/ |
| Disallow | /ex/ |
| Disallow | /orders/ |
| Disallow | /store/orders/ |
| Disallow | /redeem_coupon_token |
| Disallow | /store/order_changes/ |
| Disallow | /store/*/browse_departments* |
| Disallow | /store/*/ng/l/* |
| Disallow | /store/referrals* |
| Disallow | /store/item* |
| Disallow | /store/pickup/status/* |
| Disallow | /store/*/buy_it_again* |
| Disallow | /store/*/l/* |
| Disallow | /widgets/* |
| Disallow | /client-side-redirect/* |
| Disallow | /track/* |
| Disallow | /affiliate-program* |
| Disallow | /family/invite/* |
| Disallow | /family/accept/* |
| Disallow | /rate/orders/* |
| Disallow | /store_manager_admin |
| Disallow | /rest/v8/ |
| Disallow | /login?next= |
| Disallow | /*?*interaction_id= |
| Disallow | /*?*item_detail_product_id= |
| Disallow | /store/*/collections/*?*sisid= |
| Disallow | /store/*/*?*xrs_id= |
googlebot-mobile
| Rule | Path |
|---|---|
| Allow | / |
| Allow | /store/brands/l/pads-tampons-liners-wipes |
| Disallow | /accounts/unlock |
| Disallow | /api/ |
| Disallow | /ex/ |
| Disallow | /orders/ |
| Disallow | /store/orders/ |
| Disallow | /redeem_coupon_token |
| Disallow | /store/order_changes/ |
| Disallow | /store/*/browse_departments* |
| Disallow | /store/*/ng/l/* |
| Disallow | /store/referrals* |
| Disallow | /store/item* |
| Disallow | /store/pickup/status/* |
| Disallow | /store/*/buy_it_again* |
| Disallow | /store/*/l/* |
| Disallow | /widgets/* |
| Disallow | /client-side-redirect/* |
| Disallow | /track/* |
| Disallow | /affiliate-program* |
| Disallow | /family/invite/* |
| Disallow | /family/accept/* |
| Disallow | /rate/orders/* |
| Disallow | /store_manager_admin |
| Disallow | /rest/v8/ |
| Disallow | /login?next= |
| Disallow | /*?*interaction_id= |
| Disallow | /*?*item_detail_product_id= |
| Disallow | /store/*/collections/*?*sisid= |
| Disallow | /store/*/*?*xrs_id= |
storebot-google
| Rule | Path |
|---|---|
| Allow | / |
| Allow | /store/brands/l/pads-tampons-liners-wipes |
| Disallow | /accounts/unlock |
| Disallow | /api/ |
| Disallow | /ex/ |
| Disallow | /orders/ |
| Disallow | /store/orders/ |
| Disallow | /redeem_coupon_token |
| Disallow | /store/order_changes/ |
| Disallow | /store/*/browse_departments* |
| Disallow | /store/*/ng/l/* |
| Disallow | /store/referrals* |
| Disallow | /store/item* |
| Disallow | /store/pickup/status/* |
| Disallow | /store/*/buy_it_again* |
| Disallow | /store/*/l/* |
| Disallow | /widgets/* |
| Disallow | /client-side-redirect/* |
| Disallow | /track/* |
| Disallow | /affiliate-program* |
| Disallow | /family/invite/* |
| Disallow | /family/accept/* |
| Disallow | /rate/orders/* |
| Disallow | /store_manager_admin |
| Disallow | /rest/v8/ |
| Disallow | /login?next= |
| Disallow | /*?*interaction_id= |
| Disallow | /*?*item_detail_product_id= |
| Disallow | /store/*/collections/*?*sisid= |
| Disallow | /store/*/*?*xrs_id= |
adsbot-google-mobile
| Rule | Path |
|---|---|
| Allow | / |
| Allow | /store/brands/l/pads-tampons-liners-wipes |
| Disallow | /accounts/unlock |
| Disallow | /api/ |
| Disallow | /ex/ |
| Disallow | /orders/ |
| Disallow | /store/orders/ |
| Disallow | /redeem_coupon_token |
| Disallow | /store/order_changes/ |
| Disallow | /store/*/browse_departments* |
| Disallow | /store/*/ng/l/* |
| Disallow | /store/referrals* |
| Disallow | /store/item* |
| Disallow | /store/pickup/status/* |
| Disallow | /store/*/buy_it_again* |
| Disallow | /store/*/l/* |
| Disallow | /widgets/* |
| Disallow | /client-side-redirect/* |
| Disallow | /track/* |
| Disallow | /affiliate-program* |
| Disallow | /family/invite/* |
| Disallow | /family/accept/* |
| Disallow | /rate/orders/* |
| Disallow | /store_manager_admin |
| Disallow | /rest/v8/ |
| Disallow | /login?next= |
| Disallow | /*?*interaction_id= |
| Disallow | /*?*item_detail_product_id= |
| Disallow | /store/*/collections/*?*sisid= |
| Disallow | /store/*/*?*xrs_id= |
adsbot-google
| Rule | Path |
|---|---|
| Allow | / |
| Allow | /store/brands/l/pads-tampons-liners-wipes |
| Disallow | /accounts/unlock |
| Disallow | /api/ |
| Disallow | /ex/ |
| Disallow | /orders/ |
| Disallow | /store/orders/ |
| Disallow | /redeem_coupon_token |
| Disallow | /store/order_changes/ |
| Disallow | /store/*/browse_departments* |
| Disallow | /store/*/ng/l/* |
| Disallow | /store/referrals* |
| Disallow | /store/item* |
| Disallow | /store/pickup/status/* |
| Disallow | /store/*/buy_it_again* |
| Disallow | /store/*/l/* |
| Disallow | /widgets/* |
| Disallow | /client-side-redirect/* |
| Disallow | /track/* |
| Disallow | /affiliate-program* |
| Disallow | /family/invite/* |
| Disallow | /family/accept/* |
| Disallow | /rate/orders/* |
| Disallow | /store_manager_admin |
| Disallow | /rest/v8/ |
| Disallow | /login?next= |
| Disallow | /*?*interaction_id= |
| Disallow | /*?*item_detail_product_id= |
| Disallow | /store/*/collections/*?*sisid= |
| Disallow | /store/*/*?*xrs_id= |
bingbot
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /accounts/unlock |
| Disallow | /api/ |
| Disallow | /ex/ |
| Disallow | /orders/ |
| Disallow | /store/orders/ |
| Disallow | /redeem_coupon_token |
| Disallow | /store/order_changes/ |
| Disallow | /store/*/browse_departments* |
| Disallow | /store/*/ng/l/* |
| Disallow | /store/referrals* |
| Disallow | /store/item* |
| Disallow | /store/pickup/status/* |
| Disallow | /store/*/buy_it_again* |
| Disallow | /store/*/l/* |
| Disallow | /widgets/* |
| Disallow | /client-side-redirect/* |
| Disallow | /track/* |
| Disallow | /affiliate-program* |
| Disallow | /family/invite/* |
| Disallow | /family/accept/* |
| Disallow | /rate/orders/* |
| Disallow | /store_manager_admin |
| Disallow | /rest/v8/ |
| Disallow | /login?next= |
| Disallow | /*?*interaction_id= |
| Disallow | /*?*item_detail_product_id= |
| Disallow | /store/*/collections/*?*sisid= |
| Disallow | /store/*/*?*xrs_id=%C3%82 |
slurp
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /accounts/unlock |
| Disallow | /api/ |
| Disallow | /ex/ |
| Disallow | /orders/ |
| Disallow | /store/orders/ |
| Disallow | /redeem_coupon_token |
| Disallow | /store/order_changes/ |
| Disallow | /store/*/browse_departments* |
| Disallow | /store/*/ng/l/* |
| Disallow | /store/referrals* |
| Disallow | /store/item* |
| Disallow | /store/pickup/status/* |
| Disallow | /store/*/buy_it_again* |
| Disallow | /store/*/l/* |
| Disallow | /widgets/* |
| Disallow | /client-side-redirect/* |
| Disallow | /track/* |
| Disallow | /affiliate-program* |
| Disallow | /family/invite/* |
| Disallow | /family/accept/* |
| Disallow | /rate/orders/* |
| Disallow | /store_manager_admin |
| Disallow | /rest/v8/ |
| Disallow | /login?next= |
| Disallow | /*?*interaction_id= |
| Disallow | /*?*item_detail_product_id= |
| Disallow | /store/*/collections/*?*sisid= |
| Disallow | /store/*/*?*xrs_id= |
duckduckgo
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /accounts/unlock |
| Disallow | /api/ |
| Disallow | /ex/ |
| Disallow | /orders/ |
| Disallow | /store/orders/ |
| Disallow | /redeem_coupon_token |
| Disallow | /store/order_changes/ |
| Disallow | /store/*/browse_departments* |
| Disallow | /store/*/ng/l/* |
| Disallow | /store/referrals* |
| Disallow | /store/item* |
| Disallow | /store/pickup/status/* |
| Disallow | /store/*/buy_it_again* |
| Disallow | /store/*/l/* |
| Disallow | /widgets/* |
| Disallow | /client-side-redirect/* |
| Disallow | /track/* |
| Disallow | /affiliate-program* |
| Disallow | /family/invite/* |
| Disallow | /family/accept/* |
| Disallow | /rate/orders/* |
| Disallow | /store_manager_admin |
| Disallow | /rest/v8/ |
| Disallow | /login?next= |
| Disallow | /*?*interaction_id= |
| Disallow | /*?*item_detail_product_id= |
| Disallow | /store/*/collections/*?*sisid= |
| Disallow | /store/*/*?*xrs_id= |
oai-searchbot
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /accounts/unlock |
| Disallow | /api/ |
| Disallow | /ex/ |
| Disallow | /orders/ |
| Disallow | /store/orders/ |
| Disallow | /redeem_coupon_token |
| Disallow | /store/order_changes/ |
| Disallow | /store/*/browse_departments* |
| Disallow | /store/*/ng/l/* |
| Disallow | /store/referrals* |
| Disallow | /store/item* |
| Disallow | /store/pickup/status/* |
| Disallow | /store/*/buy_it_again* |
| Disallow | /store/*/l/* |
| Disallow | /widgets/* |
| Disallow | /client-side-redirect/* |
| Disallow | /track/* |
| Disallow | /affiliate-program* |
| Disallow | /family/invite/* |
| Disallow | /family/accept/* |
| Disallow | /rate/orders/* |
| Disallow | /store_manager_admin |
| Disallow | /rest/v8/ |
| Disallow | /login?next= |
| Disallow | /*?*interaction_id= |
| Disallow | /*?*item_detail_product_id= |
| Disallow | /store/*/collections/*?*sisid= |
| Disallow | /store/*/*?*xrs_id= |
perplexitybot
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /accounts/unlock |
| Disallow | /api/ |
| Disallow | /ex/ |
| Disallow | /orders/ |
| Disallow | /store/orders/ |
| Disallow | /redeem_coupon_token |
| Disallow | /store/order_changes/ |
| Disallow | /store/*/browse_departments* |
| Disallow | /store/*/ng/l/* |
| Disallow | /store/referrals* |
| Disallow | /store/item* |
| Disallow | /store/pickup/status/* |
| Disallow | /store/*/buy_it_again* |
| Disallow | /store/*/l/* |
| Disallow | /widgets/* |
| Disallow | /client-side-redirect/* |
| Disallow | /track/* |
| Disallow | /affiliate-program* |
| Disallow | /family/invite/* |
| Disallow | /family/accept/* |
| Disallow | /rate/orders/* |
| Disallow | /store_manager_admin |
| Disallow | /rest/v8/ |
| Disallow | /login?next= |
| Disallow | /*?*interaction_id= |
| Disallow | /*?*item_detail_product_id= |
| Disallow | /store/*/collections/*?*sisid= |
| Disallow | /store/*/*?*xrs_id= |
applebot
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /accounts/unlock |
| Disallow | /api/ |
| Disallow | /ex/ |
| Disallow | /orders/ |
| Disallow | /store/orders/ |
| Disallow | /redeem_coupon_token |
| Disallow | /store/order_changes/ |
| Disallow | /store/*/browse_departments* |
| Disallow | /store/*/ng/l/* |
| Disallow | /store/referrals* |
| Disallow | /store/item* |
| Disallow | /store/pickup/status/* |
| Disallow | /store/*/buy_it_again* |
| Disallow | /store/*/l/* |
| Disallow | /widgets/* |
| Disallow | /client-side-redirect/* |
| Disallow | /track/* |
| Disallow | /affiliate-program* |
| Disallow | /family/invite/* |
| Disallow | /family/accept/* |
| Disallow | /rate/orders/* |
| Disallow | /store_manager_admin |
| Disallow | /rest/v8/ |
| Disallow | /login?next= |
| Disallow | /*?*interaction_id= |
| Disallow | /*?*item_detail_product_id= |
| Disallow | /store/*/collections/*?*sisid= |
| Disallow | /store/*/*?*xrs_id= |
linkedinbot
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /accounts/unlock |
| Disallow | /api/ |
| Disallow | /ex/ |
| Disallow | /orders/ |
| Disallow | /store/orders/ |
| Disallow | /redeem_coupon_token |
| Disallow | /store/order_changes/ |
| Disallow | /store/*/browse_departments* |
| Disallow | /store/*/ng/l/* |
| Disallow | /store/referrals* |
| Disallow | /store/item* |
| Disallow | /store/pickup/status/* |
| Disallow | /store/*/buy_it_again* |
| Disallow | /store/*/l/* |
| Disallow | /widgets/* |
| Disallow | /client-side-redirect/* |
| Disallow | /track/* |
| Disallow | /affiliate-program* |
| Disallow | /family/invite/* |
| Disallow | /family/accept/* |
| Disallow | /rate/orders/* |
| Disallow | /store_manager_admin |
| Disallow | /rest/v8/ |
| Disallow | /login?next= |
| Disallow | /*?*interaction_id= |
| Disallow | /*?*item_detail_product_id= |
| Disallow | /store/*/collections/*?*sisid= |
| Disallow | /store/*/*?*xrs_id= |
twitterbot
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /accounts/unlock |
| Disallow | /api/ |
| Disallow | /ex/ |
| Disallow | /orders/ |
| Disallow | /store/orders/ |
| Disallow | /redeem_coupon_token |
| Disallow | /store/order_changes/ |
| Disallow | /store/*/browse_departments* |
| Disallow | /store/*/ng/l/* |
| Disallow | /store/referrals* |
| Disallow | /store/item* |
| Disallow | /store/pickup/status/* |
| Disallow | /store/*/buy_it_again* |
| Disallow | /store/*/l/* |
| Disallow | /widgets/* |
| Disallow | /client-side-redirect/* |
| Disallow | /track/* |
| Disallow | /affiliate-program* |
| Disallow | /family/invite/* |
| Disallow | /family/accept/* |
| Disallow | /rate/orders/* |
| Disallow | /store_manager_admin |
| Disallow | /rest/v8/ |
| Disallow | /login?next= |
| Disallow | /*?*interaction_id= |
| Disallow | /*?*item_detail_product_id= |
| Disallow | /store/*/collections/*?*sisid= |
| Disallow | /store/*/*?*xrs_id= |
slackbot-linkexpanding
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /accounts/unlock |
| Disallow | /api/ |
| Disallow | /ex/ |
| Disallow | /orders/ |
| Disallow | /store/orders/ |
| Disallow | /redeem_coupon_token |
| Disallow | /store/order_changes/ |
| Disallow | /store/*/browse_departments* |
| Disallow | /store/*/ng/l/* |
| Disallow | /store/referrals* |
| Disallow | /store/item* |
| Disallow | /store/pickup/status/* |
| Disallow | /store/*/buy_it_again* |
| Disallow | /store/*/l/* |
| Disallow | /widgets/* |
| Disallow | /client-side-redirect/* |
| Disallow | /track/* |
| Disallow | /affiliate-program* |
| Disallow | /family/invite/* |
| Disallow | /family/accept/* |
| Disallow | /rate/orders/* |
| Disallow | /store_manager_admin |
| Disallow | /rest/v8/ |
| Disallow | /login?next= |
| Disallow | /*?*interaction_id= |
| Disallow | /*?*item_detail_product_id= |
| Disallow | /store/*/collections/*?*sisid= |
| Disallow | /store/*/*?*xrs_id= |
*
| Rule | Path |
|---|---|
| Disallow | / |
Comments