theaquarian.com
robots.txt
Robots Exclusion Standard data for theaquarian.com
Resource Scan
Scan Details
| Site Domain | theaquarian.com |
| Base Domain | theaquarian.com |
| Scan Status | Ok |
| Last Scan | 2025-05-25T19:20:06+00:00 |
| Next Scan | 2025-06-24T19:20:06+00:00 |
Last Scan
| Scanned | 2025-05-25T19:20:06+00:00 |
| URL | https://theaquarian.com/robots.txt |
| Domain IPs | 69.164.215.69 |
| Response IP | 69.164.215.69 |
| Found | Yes |
| Hash | 7ca2a775d67594f699f9be6c5f6a151c5c90cf9b8b8a525c4f509708408c28f2 |
| SimHash | 7c468911cee3 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /wp-admin/ |
| Allow | /wp-admin/admin-ajax.php |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
adsbot-google
agentic
ai article writer
ai content detector
ai dungeon
ai search engine
ai seo crawler
ai training
ai writer
ai21 labs
ai2bot
ai2bot-dolma
aibot
aihitbot
aimatrix
aisearchbot
ai training
aitraining
alexa
alpha ai
alphaai
amazon bedrock
amazon comprehend
amazon lex
amazon sagemaker
amazon silk
amazon textract
amazon-kendra
amazonbot
amelia
anderspinkbot
anthropic
anthropic-ai
anypicker
anyword
applebot
applebot-extended
aria browse
articoolo
automated writer
awariorssbot
awariosmartbot
azure
bardbot
brave leo
brightbot 1.0
bytedance
bytespider
catboost
cc-crawler
ccbot
chatglm
chatgpt-user
chinchilla
claude
claude-web
claudebot
clearscope
cohere
cohere-ai
cohere-training-data-crawler
common crawl
commoncrawl
content harmony
content king
content optimizer
content samurai
contentatscale
contentbot
contentedge
conversion ai
copilot
copyai
copymatic
copyscape
cotoyogi
crawlq ai
crawlspace
crew ai
crewai
dall-e
dataforseobot
dataprovider
deepai
deepl
deepmind
deepseek
diffbot
doubao ai
duckassistbot
factset_spyderbot
falcon
firecrawl
firecrawlagent
flyriver
frase ai
friendlycrawler
gemini
gemma
genai
genspark
glm
google-extended
googleother
googleother-image
googleother-video
goose
gpt
gptbot
grammarly
grendizer
grok
gt bot
gtbot
hemingway editor
hugging face
hypotenuse ai
iaskspider
iaskspider/2.0
icc-crawler
imagegen
imagesiftbot
img2dataset
imgproxy
inferkit
ink editor
inkforall
intelliseek
inferkit
isscyberriskcrawler
jasperai
kafkai
kangaroo
kangaroo bot
keyword density ai
knowledge
komobot
llama
llms
magpie-crawler
marketmuse
meltwater
meta ai
meta-ai
metaai
metatagbot
mistral
narrative
neevabot
neural text
neuralseo
nova act
novaact
oai-searchbot
omgili
omgilibot
open ai
openai
openbot
opentext ai
operator
outwrite
page analyzer ai
pangubot
paperlibot
paraphraser.io
peer39_crawler
perplexity
perplexity-user
perplexitybot
petalbot
phindbot
piplbot
prowritingaid
quillbot
robotspider
rytr
saplingai
scalenut
scraper
scrapy
scriptbook
seekr
semrushbot
semrushbot-ba
semrushbot-ft
semrushbot-ocob
semrushbot-si
semrushbot-swa
sentibot
seo content machine
seo robot
sidetrade
sidetrade indexer bot
simplified ai
sitefinity
skydancer
slickwrite
sonic
spin rewriter
spinbot
stability
stablediffusionbot
sudowrite
summalybot
super agent
surfer ai
text blaze
textcortex
the knowledge ai
tiktokspider
timpibot
velenpublicwebcrawler
vidnami ai
webzio
webzio-extended
whisper
wordai
wordtune
wormsgtp
wpbot
writecream
writerzen
writescope
writesonic
xai
xbot
youbot
zero gtp
zerochat
zhipu
zimm
adidxbot
ahrefsbot
ahrefssiteaudit
aliyunsecbot
apache-httpclient
audisto crawler
awariosmartbot
baiduspider
barkrowler
bingbot
bingbot
blexbot
bravebot
buck
coccocbot-image
coccocbot-web
dnbcrawler
dotbot
dotbot
duckassistbot
fluid
flyriverbot
grammarly
hubspot crawler
iframely
linkchecker
mj12bot
moreover
orbbot
petalbot
scrapy
screaming frog seo spider
semanticscholarbot
seznambot
siteauditbot
slurp
summalybot
trendictionbot
turnitin
twingly recon
yacybot
yak
yandexbot
yandeximages
yeti
zoombot
| Rule | Path |
|---|---|
| Disallow | / |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.theaquarian.com/wp-sitemap.xml |
Warnings
- 1 invalid line.
Comments