shopwoodmans.com
robots.txt

Robots Exclusion Standard data for shopwoodmans.com

Resource Scan

Scan Details

Site Domain shopwoodmans.com
Base Domain shopwoodmans.com
Scan Status Ok
Last Scan2024-09-11T21:29:33+00:00
Next Scan 2024-09-25T21:29:33+00:00

Last Scan

Scanned2024-09-11T21:29:33+00:00
URL https://shopwoodmans.com/robots.txt
Domain IPs 13.33.30.15, 13.33.30.48, 13.33.30.5, 13.33.30.67
Response IP 13.33.30.67
Found Yes
Hash 7e3f4b3a2dc8f4143935ff771238033c0217218e9ff4313401a90e2a7505c9aa
SimHash d57654320367

Groups

yandex

Rule Path
Disallow /

facebookexternalhit

Rule Path
Disallow /assets

*

Rule Path
Disallow /api/
Disallow /ex/
Disallow /orders/
Disallow /client-side-redirect/*
Allow /store/

Comments

  • Any bot scraping or crawling this site must abide by
  • Instacart's Terms of Service https://www.instacart.com/terms