Log In Sign Up

scratchpad.com
robots.txt

Robots Exclusion Standard data for scratchpad.com

Resource Scan

Scan Details

Site Domain scratchpad.com
Base Domain scratchpad.com
Scan Status Ok
Last Scan2025-08-28T07:14:10+00:00
Next Scan 2025-09-27T07:14:10+00:00

Last Scan

Scanned2025-08-28T07:14:10+00:00
URL https://scratchpad.com/robots.txt
Redirect https://www.scratchpad.com/robots.txt
Redirect Domain www.scratchpad.com
Redirect Base scratchpad.com
Domain IPs 13.203.125.58, 13.233.175.166, 3.109.243.18
Redirect IPs 13.203.125.58, 13.233.175.166, 3.109.243.18
Response IP 52.68.134.190
Found Yes
Hash 7d90242e2408836035b8958123e01aebe9b41d8edeaed948f7e0d10ac65996ce
SimHash 515cff40cd13

Groups

*

Rule Path
Disallow /accept-invite/
Disallow /login/
Disallow /signup/
Disallow /api/
Disallow /static/
Disallow /404

gptbot

Rule Path
Allow /

claudebot

Rule Path
Allow /

perplexitybot

Rule Path
Allow /

google-extended

Rule Path
Allow /

ccbot

Rule Path
Allow /

gptbot

Rule Path
Disallow /accept-invite/
Disallow /login/
Disallow /signup/
Back to top

Other Records

Field Value
sitemap https://www.scratchpad.com/sitemap.xml
Back to top

Comments

  • Allow OpenAI's GPTBot (ChatGPT, etc.)
  • Allow Anthropic’s Claude
  • Allow Perplexity’s crawler
  • Allow Google's AI tools (Bard, SGE, etc.)
  • Allow CommonCrawl
  • Disallow all AI crawlers from non-public or utility paths
Back to top