Log In Sign Up

brandefense.io
robots.txt

Robots Exclusion Standard data for brandefense.io

Resource Scan

Scan Details

Site Domain brandefense.io
Base Domain brandefense.io
Scan Status Ok
Last Scan2026-04-07T05:40:38+00:00
Next Scan 2026-04-21T05:40:38+00:00

Last Scan

Scanned2026-04-07T05:40:38+00:00
URL https://brandefense.io/robots.txt
Domain IPs 104.18.14.124, 104.18.15.124, 2606:4700::6812:e7c, 2606:4700::6812:f7c
Response IP 104.18.14.124
Found Yes
Hash 6b2a9c09c782c4f8b090d285e04d2d5a3102a98704a52d44633703aa78f0a58f
SimHash 13091b01ebf3

Groups

*

Rule Path
Disallow *?p=
Disallow *%26p%3D
Disallow *?s=
Disallow *%26s%3D
Disallow /?
Disallow /wp-content/plugins/
Disallow /wp-admin/
Disallow /wp-login.php
Disallow /ajax*
Disallow /*__hstc*
Disallow /wp-json*
Disallow *?*utm*=
Disallow *?*openstat=
Disallow *?*from=
Disallow *?*yclid=
Disallow *?*ymclid=
Disallow *?*gclid=
Disallow *?doing_wp_cron=
Allow /wp-admin/admin-ajax.php
Allow /feed/
Allow /

gptbot

Rule Path
Allow /

chatgpt-user

Rule Path
Allow /

google-extended

Rule Path
Allow /

perplexitybot

Rule Path
Allow /

ccbot

Rule Path
Allow /

claudebot

Rule Path
Allow /

anthropic-ai

Rule Path
Allow /

applebot-extended

Rule Path
Allow /
Back to top

Other Records

Field Value
sitemap https://brandefense.io/sitemap_index.xml
sitemap https://brandefense.io/post-sitemap1.xml
sitemap https://brandefense.io/post-sitemap2.xml
sitemap https://brandefense.io/post-sitemap3.xml
sitemap https://brandefense.io/page-sitemap.xml
Back to top

Comments

  • =====================================
  • Brandefense.io Robots.txt
  • Optimized for SEO + AI Crawlers
  • =====================================
  • =====================================
  • AI / LLM Crawlers – Explicit Permission
  • =====================================
  • =====================================
  • LLM Policy Reference
  • =====================================
  • =====================================
  • Sitemap Index
  • =====================================
Back to top

Warnings

  • `llms` is not a known field.
Back to top