sameday.costco.com
robots.txt

Robots Exclusion Standard data for sameday.costco.com

Resource Scan

Scan Details

Site Domain sameday.costco.com
Base Domain costco.com
Scan Status Ok
Last Scan2024-06-01T20:57:29+00:00
Next Scan 2024-06-15T20:57:29+00:00

Last Scan

Scanned2024-06-01T20:57:29+00:00
URL https://sameday.costco.com/robots.txt
Domain IPs 108.157.254.112, 108.157.254.43, 108.157.254.53, 108.157.254.75
Response IP 108.157.254.75
Found Yes
Hash cb353801368f61de31bdac3b0454ac8e84413bbf776591e0547b1f96ec7210f7
SimHash 977654220366

Groups

yandex

Rule Path
Disallow /

facebookexternalhit

Rule Path
Disallow /assets

*

Rule Path
Disallow /api/
Disallow /store/
Disallow /ex/
Disallow /orders/
Disallow /client-side-redirect/*

Comments

  • Any bot scraping or crawling this site must abide by
  • Instacart's Terms of Service https://www.instacart.com/terms