Log In Sign Up

bra.pe
robots.txt

Robots Exclusion Standard data for bra.pe

Resource Scan

Scan Details

Site Domain bra.pe
Base Domain bra.pe
Scan Status Ok
Last Scan2025-10-16T04:16:16+00:00
Next Scan 2025-11-15T04:16:16+00:00

Last Scan

Scanned2025-10-16T04:16:16+00:00
URL http://bra.pe/robots.txt
Redirect https://www.bratabase.com//robots.txt
Redirect Domain www.bratabase.com
Redirect Base bratabase.com
Domain IPs 45.79.141.117
Redirect IPs 45.79.141.117
Response IP 45.79.141.117
Found Yes
Hash 5ad44dee3fb806ce893ad1642d22aa46378b7a3157c878af429bb9f784d9e448
SimHash f6d75840c3d4

Groups

googlebot-image

Rule Path
Disallow /cache/*.jpg$

*

Rule Path Comment
Disallow /chest/* -
Disallow /accounts/* -
Disallow /search/* -
Disallow /tag/* -
Disallow /cache/*.jpg$ -
Disallow /browse/*/*/mentions/$ -
Disallow /browse/*/*/lists/$ -
Disallow /browse/*/*/listings/$ -
Disallow /browse/*/*/*/explore/$ has noindex
Disallow /browse/*/*/issues/$ -
Disallow /browse/*/activity/$ -
Disallow /browse/*/related/$ -
Disallow /browse/*/tag-models/$ -
Disallow /browse/*/vs/brand/$ -
Disallow /profile/*/participation/$ -
Disallow /profile/*/lists/$ -
Disallow /profile/*/follow*/$ -
Disallow /go-deeper/ -

dataforseobot
blexbot
buck
neevabot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 99

gptbot
claudebot
claude-web
ccbot
googlebot-extended
applebot-extended
facebookbot
meta-externalagent
meta-externalfetcher
diffbot
perplexitybot
omgili
omgilibot
webzio-extended
imagesiftbot
bytespider
amazonbot
youbot
semrushbot-ocob
petalbot
velenpublicwebcrawler
turnitinbot
timpibot
oai-searchbot
icc-crawler
ai2bot
ai2bot-dolma
dataforseobot
awariobot
awariosmartbot
awariorssbot
google-cloudvertexbot
pangubot
kangaroo bot
sentibot
img2dataset
meltwater
seekr
peer39_crawler
cohere-ai
cohere-training-data-crawler
duckassistbot
scrapy

Rule Path
Disallow /

*

No rules defined. All paths allowed.

Back to top

Comments

  • Block all known AI crawlers and assistants
  • from using content for training AI models.
  • Source: https://robotstxt.com/ai
  • Block any non-specified AI crawlers (e.g., new
  • or unknown bots) from using content for training
  • AI models. This directive is still experimental
  • and may not be supported by all AI crawlers.
Back to top

Warnings

  • `disallowaitraining` is not a known field.
Back to top