dre.tretas.org
robots.txt
Robots Exclusion Standard data for dre.tretas.org
Resource Scan
Scan Details
| Site Domain | dre.tretas.org |
| Base Domain | tretas.org |
| Scan Status | Ok |
| Last Scan | 2026-02-19T15:57:43+00:00 |
| Next Scan | 2026-03-21T15:57:43+00:00 |
Last Scan
| Scanned | 2026-02-19T15:57:43+00:00 |
| URL | https://dre.tretas.org/robots.txt |
| Domain IPs | 148.69.115.212 |
| Response IP | 148.69.115.212 |
| Found | Yes |
| Hash | d18780f757eca89334ef8a64617c91421e79f1651694c2d6d28fc55261b6cabf |
| SimHash | c176d08bccd6 |
Groups
*
| Rule | Path |
|---|---|
| Allow | /sitemap-documents.xml?p=* |
| Allow | /static/css/style.css?v=* |
| Allow | /static/css/style_normal.css?v=* |
| Allow | /static/css/style_mobile.css?v=* |
| Disallow | /pdfs/ |
| Disallow | |
| Disallow | /registar |
| Disallow | /auth |
| Disallow | *.json/ |
| Disallow | *.json |
| Disallow | */related/ |
arquivo-web-crawler
| Rule | Path |
|---|---|
| Allow | /sitemap-documents.xml?p=* |
| Allow | /static/css/style.css?v=* |
| Allow | /static/css/style_normal.css?v=* |
| Allow | /static/css/style_mobile.css?v=* |
| Disallow | /pdfs/ |
| Disallow | |
| Disallow | /registar |
| Disallow | /auth |
| Disallow | */related/ |
msnbot
| Rule | Path |
|---|---|
| Allow | /sitemap-documents.xml?p=* |
| Disallow | /pdfs/ |
| Disallow | |
| Disallow | /register |
| Disallow | /auth |
| Disallow | *.json/ |
| Disallow | *.json |
| Disallow | */related/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 5 |
bingbot
| Rule | Path |
|---|---|
| Allow | /sitemap-documents.xml?p=* |
| Disallow | /pdfs/ |
| Disallow | |
| Disallow | /register |
| Disallow | /auth |
| Disallow | *.json/ |
| Disallow | *.json |
| Disallow | */related/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 5 |
*
| Rule | Path |
|---|---|
| Disallow |
-ai
_ai
ai.
ai-
ai_
ai=
addsearchbot
agentic
agentql
agent 3
agent api
agent-
ai agent
ai article writer
ai assistant
ai chat
ai content detector
ai detection
ai dungeon
ai journalist
ai legion
ai rag
ai search
ai seo crawler
ai training
ai web
ai writer
ai2
aibot
aihitbot
aimatrix
airesearch
aisearch
aitraining
alexa
alice yandex
aligenie
aliyunsec
alpha ai
alphaai
amazon
amelia
anderspinkbot
andibot
anomura
anonymous ai
anthropic
anypicker
anyword
applebot
aria ai
aria browse
articoolo
ask ai
atlassian
autogen
automated writer
automl
autonomous rag
autorag
awario
aws trainium
azure
babyagi
babycatagi
bardbot
basic rag
bedrock
big sur
bigsur
botsonic
brightbot
browser mcp agent
browser use
buddybot
bytebot
bytedance
bytespider
carynai
catboost
cc-crawler
ccbot
chai
channel3bot
character
charstar ai
chatbot
chatsonic
chatuser
chinchilla
claude
clearscope
clearview
cognitive ai
cohere
common crawl
commoncrawl
compass
content harmony
content king
content optimizer
content samurai
contentatscale
contentbot
contentedge
contentshake
conversion ai
copilot
copyai
copymatic
copyscape
coreweave
corrective rag
cotoyogi
crab
crawl4ai
crawlq ai
crawlspace
crew ai
crewai
crushon ai
cypex
dall-e
darkbard
darkbert
datafor
dataprovider
datenbank crawler
deepai
deep ai
deepl
deepmind
deep research
deepresearch
deepseek
devin
diffbot
dolma
doubao ai
duckassistbot
duckduckgo chat
duckduckgo-enhanced
echobot
echobox
elixir
facebookbot
facebookexternalhit
factset
falcon
fire-1
firebase
firecrawl
flyriver
frase ai
friendlycrawler
gato
gemini
gemma
gen ai
genai
generative
genspark
gentoo-chat
ghostwriter
gigachat
glm
godmode
goose
gpt
grammarly
grendizer
grok
groq
gt bot
gtbot
gtp
hemingway editor
hetzner
hugging
hunyuan
hybrid search rag
hypotenuse ai
iask
iboubot
icc-crawler
ideogram
imagegen
imagen
imagesiftbot
imagespider
img2dataset
imgproxy
ink editor
inkforall
instructor
intelliseek
inferkit
isscyberriskcrawler
janitor ai
jasper
jenni ai
julius ai
kafkai
kaggle
kangaroo
keyword density ai
kimi
klaviyo
knowledge
komobot
kruti
kunatocrawler
laiondownloader
langchain
langfuse
language ai
le chat
lensa
leonardo
lightpanda
lightrag
linerbot
linkupbot
llama
lcc
llm
local rag agent
lovable
magistral
magpie-crawler
mai-image
manus
marketmuse
mbzuai
meltwater
meta-ai
meta-external
meta-webindexer
meta ai
metaai
metatagbot
middleware
midjourney
mini agi
minimax
mintlify
mistral
mixtral
model-training
molmo
monica
nano
narrative
neevabot
netestate
neural text
neuralseo
ninjaai
nodezero
notebooklm
nova act
novaact
nytheon
oai-searchbot
oai searchbot
oasis
olivia
olm
omgili
onyx
open ai
open interpreter
openagi
openai
openbot
openinterpreter
openpi
openrouter
opentext ai
operator
outwrite
ovhcloud
owlin
page analyzer ai
palm 2
pangubot
panscient
paperlibot
paraphraser.io
peer39_crawler
perflexity
perplexity
petal
phind
piplbot
pixmo
playwright
poebot
poesearchbot
poggio
poseidon
prowritingaid
proximic
pulsarrpa
puppeteer
python ai
python lxml
pytorch
qualified
quark
quillbot
qopywriter
qwen
rag agent
rag azure ai
rag chatbot
rag database
rag is
rag pipeline
rag search
rag with
rag-
rag_
raptor
react agent
reasoning
redis ai rag
reka flash
replicate
robotspider
runpod
rytr
saplingai
sbintuitionsbot
scala
scalenut
scrap
scriptbook
seekr
seobot
seo content machine
seo robot
semrushbot
sentibot
serper
shapbot
sidetrade
simplified ai
sitefinity
skydancer
skyvern
slickwrite
smartbot
sonic
sora
spider/2
spidercreator
spin rewrite
spinbot
stability
stable diffusion
stablediffusionbot
stagehand
stealth
sudowrite
summalybot
super agent
superagent
superagi
surfer ai
taiko
tavilybot
terracotta
text blaze
textcortex
thinkbot
thordata
tiktokspider
timpibot
tinybird
together
traefik
tulu
tã¼lu
turnitinbot
twinagent
uagents
ui-tars
velenpublicwebcrawler
venus chub ai
vidnami ai
vision rag
wardbot
websurfer
webtext
webvoyager
webzio
wechat
whisper
wordai
wordtune
wpbot
writecream
writerzen
writescope
writesonic
wrtnbot
xai
xanthorox
xbot
xlnet
yak
yaml
yandexadditional
youbot
zanista
zendesk
zero
zhipu
zhuque ai
zimm
| Rule | Path |
|---|---|
| Disallow | / |
Other Records
| Field | Value |
|---|---|
| sitemap | http://dre.tretas.org/sitemap.xml |
Warnings
- 1 invalid line.
Comments