Log In Sign Up

brave.org
robots.txt

Robots Exclusion Standard data for brave.org

Resource Scan

Scan Details

Site Domain brave.org
Base Domain brave.org
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-08-05T04:41:22+00:00
Next Scan 2025-10-04T04:41:22+00:00

Last Successful Scan

Scanned2025-05-31T04:17:29+00:00
URL https://brave.org/robots.txt
Domain IPs 13.85.85.117
Response IP 13.85.85.117
Found Yes
Hash 846872c27ef237368430a2a56cc55e03f53f7a190d57a9157666c2a5ad61cbfc
SimHash d11ec9106fd8

Groups

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

openai-search

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

duckduckbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

zoombot

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

coccocbot-image

Rule Path
Disallow /

awariobot

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

wpbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

googlebot

Rule Path
Allow /

bingbot

Rule Path
Allow /

*

Rule Path
Allow /
Back to top

Comments

  • Block SEO/data-mining bots
  • Allow useful bots
Back to top

Warnings

  • 2 invalid lines.
Back to top