Log In Sign Up

shopify.dev
robots.txt

Robots Exclusion Standard data for shopify.dev

Resource Scan

Scan Details

Site Domain shopify.dev
Base Domain shopify.dev
Scan Status Ok
Last Scan2025-08-27T20:09:44+00:00
Next Scan 2025-09-26T20:09:44+00:00

Last Scan

Scanned2025-08-27T20:09:44+00:00
URL https://shopify.dev/robots.txt
Domain IPs 185.146.173.20
Response IP 185.146.173.20
Found Yes
Hash 1b0b46ff4abadecc025e05309a3e90929d21021bd9fee86c2576c42196b18a12
SimHash 88354924bd75

Groups

*

Rule Path
Disallow /*?*shpxid=*
Disallow /beta/
Disallow /workshops/
Disallow /api/shipping-partner-platform/
Disallow /docs/api/shipping-partner-platform/

ccbot

Rule Path
Disallow /apps/default-app-home

chatgpt-user

Rule Path
Disallow /apps/default-app-home
Back to top

Other Records

Field Value
sitemap https://shopify.dev/sitemap.xml
Back to top

Comments

  • See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file
  • For any LLM training, we have implemented https://shopify.dev/llms.txt following https://llmstxt.org/ guidelines. You can append .txt to the end of any URL to get the raw text version of the page.
  • disallow Common Crawl bot in effort to prevent being added to the Common Crawl dataset (used in GPT training)
  • disallow ChatGPT plugins from accessing certain routes
Back to top