Log In Sign Up

kopperfield.com
robots.txt

Robots Exclusion Standard data for kopperfield.com

Resource Scan

Scan Details

Site Domain kopperfield.com
Base Domain kopperfield.com
Scan Status Ok
Last Scan2026-02-17T16:42:12+00:00
Next Scan 2026-03-03T16:42:12+00:00

Last Scan

Scanned2026-02-17T16:42:12+00:00
URL https://kopperfield.com/robots.txt
Redirect https://www.kopperfield.com/robots.txt
Redirect Domain www.kopperfield.com
Redirect Base kopperfield.com
Domain IPs 76.76.21.21
Redirect IPs 66.33.60.66, 76.76.21.142
Response IP 76.76.21.123
Found Yes
Hash af5206b16ffb1e38417260877e30e166de340f93bfb9105f8f5f6c12174b1afb
SimHash 6b149816cc95

Groups

*

Rule Path
Disallow /api/
Disallow /admin/
Disallow /order/
Disallow /dashboard/
Disallow /provider/
Disallow /lp/princeton-electrical-makeover
Disallow /tesla/
Disallow /partner/
Disallow /ecm
Disallow /home-charging

Other Records

Field Value
crawl-delay 10

gptbot

Rule Path
Allow /
Disallow /api/
Disallow /admin/
Disallow /dashboard/
Disallow /provider/
Disallow /partner/

Other Records

Field Value
crawl-delay 30

claude-web

Rule Path
Allow /
Disallow /api/
Disallow /admin/
Disallow /dashboard/
Disallow /provider/
Disallow /partner/

Other Records

Field Value
crawl-delay 30

claudebot

Rule Path
Allow /
Disallow /api/
Disallow /admin/
Disallow /dashboard/
Disallow /provider/
Disallow /partner/

Other Records

Field Value
crawl-delay 30

google-extended

Rule Path
Allow /
Disallow /api/
Disallow /admin/
Disallow /dashboard/
Disallow /provider/
Disallow /partner/

Other Records

Field Value
crawl-delay 30

perplexitybot

Rule Path
Allow /
Disallow /api/
Disallow /admin/
Disallow /dashboard/
Disallow /provider/
Disallow /partner/

Other Records

Field Value
crawl-delay 30

ccbot

Rule Path
Allow /
Disallow /api/
Disallow /admin/
Disallow /dashboard/
Disallow /provider/
Disallow /partner/

Other Records

Field Value
crawl-delay 30
Back to top

Other Records

Field Value
sitemap https://www.kopperfield.com/sitemap.xml
Back to top

Comments

  • Default rules for all crawlers
  • AI Crawlers - Explicitly allow access to public content for GEO
  • OpenAI (ChatGPT)
  • Anthropic (Claude)
  • Google (Gemini/Bard)
  • Perplexity AI
  • Common Crawl (used by many AI models)
Back to top