correiocidadania.com.br
robots.txt

Groups

agent gpt
agentgpt
ai article writer
ai content detector
ai dungeon
ai search engine
ai seo crawler
ai writer
ai21 labs
ai2bot
aibot
aisearchbot
alexatm
alpha ai
alphaai
amazon bedrock
amazon lex
amazonbot
amelia
anthropic-ai
anthropicai
anypicker
anyword
applebot
articoolo
autogpt
automated writer
awariorssbot
awariosmartbot
bingai
brave leo ai
bytespider
catboost
cc-crawler
ccbot
chatgpt
chinchilla
claude-web
claudebot
clearscope
cohere-ai
cohere-training-data-crawler
common crawl
commoncrawl
content harmony
content king
content optimizer
content samurai
contentatscale
contentbot
contentedge
conversion ai
copyai
copymatic
copyscape
crawlq ai
crawlspace
crew ai
crewai
dall-e
dataforseobot
deepai
deepl
deepmind
deepseek
depolarizinggpt
dialogpt
diffbot
duckassistbot
facebookbot
firecrawl
flyriver
frase ai
friendlycrawler
gemini
gemma
genai
google bard ai
google-cloudvertexbot
google-extended
googleother
gpt-2
gpt-3
gpt-4
gptbot
gptzero
grammarly
grok
hemingway editor
hugging face
hypotenuse ai
iaskspider
icc-crawler
imagesiftbot
img2dataset
ink editor
inkforall
intelliseek.ai
inferkit
isscyberriskcrawler
jasperai
kafkai
kangaroo
keyword density ai
leftwinggpt
llama
magpie-crawler
marketmuse
meltwater
meta ai
meta llama
meta.ai
meta-ai
meta-externalagent
meta-externalfetcher
metaai
metatagbot
mistral
narrative device
neural text
neuralseo
oai-searchbot
oai searchbot
omgili
omnigpt
open ai
openai
opentext ai
outwrite
page analyzer ai
pangubot
paraphraser.io
peer39_crawler
perplexitybot
petalbot
prowritingaid
quillbot
rightwinggpt
robotspider
rytr
saplingai
scalenut
scrapy
scriptbook
searchgpt
semrushbot
seo content machine
seo robot
sidetrade
simplified ai
slickwrite
spin rewriter
spinbot
stability
sudowrite
surfer ai
text blaze
textcortex
the knowledge ai
timpibot
velenpublicwebcrawler
vidnami ai
webchatgpt
webzio
whisper
wordai
wordtune
writecream
writerzen
writescope
writesonic
x.ai
xai
youbot
zero gtp
zimmwriter

femtosearchbot
serpstatbot

googlebot
slurp
msnbot
mediapartners-google*
googlebot-image
yahoo-mmcrawler

Other Records

*

Comments

• If the Joomla site is installed within a folder such as at
• e.g. www.example.com/joomla/ the robots.txt file MUST be
• moved to the site root at e.g. www.example.com/robots.txt
• AND the joomla folder name MUST be prefixed to the disallowed
• path, e.g. the Disallow rule for the /administrator/ folder
• MUST be changed to read Disallow: /joomla/administrator/
• For more information about the robots.txt standard, see:
• http://www.robotstxt.org/orig.html
• For syntax checking, see:
• http://tool.motoricerca.info/robots-checker.phtml
• Ultimate AI Block List v1.2 20250212
• https://perishablepress.com/ultimate-ai-block-list/
• bots bloqueados
• bots liberados