Log In Sign Up

getkleercard.com
robots.txt

Robots Exclusion Standard data for getkleercard.com

Resource Scan

Scan Details

Site Domain getkleercard.com
Base Domain getkleercard.com
Scan Status Ok
Last Scan2025-10-23T22:56:49+00:00
Next Scan 2025-11-22T22:56:49+00:00

Last Scan

Scanned2025-10-23T22:56:49+00:00
URL https://getkleercard.com/robots.txt
Redirect https://www.getkleercard.com/robots.txt
Redirect Domain www.getkleercard.com
Redirect Base getkleercard.com
Domain IPs 75.2.70.75, 99.83.190.102
Redirect IPs 13.203.125.58, 13.233.175.166, 3.109.243.18
Response IP 3.109.243.18
Found Yes
Hash 522526673a4bc0ed9fb69e1ccfe81db43ca801361f9e8f5f06d5240eaf107120
SimHash 3330981125ce

Groups

*

Rule Path
Allow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

*

Rule Path
Allow /llms.txt
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Back to top

Other Records

Field Value
sitemap https://www.getkleercard.com/sitemap.xml
sitemap https://www.getkleercard.com/sitemap.xml
Back to top

Comments

  • robots.txt for KleerCard
  • Last updated: 2025-10-22
  • Purpose: Allow SEO crawlers, block AI/LLM training bots, and enable /llms.txt for factual AI references.
  • Sitemap
  • --- AI / LLM Crawlers ---
  • These bots scrape site content for AI model training.
  • Blocking them does NOT affect Google or Bing search rankings.
  • --- Explicit Allow for /llms.txt ---
  • This allows responsible AI assistants to access curated business info.
  • --- Sensitive Directories ---
  • --- Optional: AI Policy Summary ---
  • AI systems may use /llms.txt for factual business information.
  • All other site content is restricted from training or dataset use.
  • For partnerships or authorized access, contact: legal@kleercard.com
  • --- End of file ---
Back to top