Log In Sign Up

mywebmaestro.com
robots.txt

Robots Exclusion Standard data for mywebmaestro.com

Resource Scan

Scan Details

Site Domain mywebmaestro.com
Base Domain mywebmaestro.com
Scan Status Ok
Last Scan2025-08-16T16:34:32+00:00
Next Scan 2025-09-15T16:34:32+00:00

Last Scan

Scanned2025-08-16T16:34:32+00:00
URL https://mywebmaestro.com/robots.txt
Domain IPs 65.60.38.74
Response IP 65.60.38.74
Found Yes
Hash 3fb937cbc94375758ec252f176a919b0d05ee5a11496cd5154b73f7f9f604be3
SimHash 6866980d9cd3

Groups

*

Rule Path
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /cgi-bin/
Disallow /trackback/
Disallow /xmlrpc.php
Disallow /wp-login.php
Disallow /wp-register.php
Disallow /?s=
Allow /wp-content/uploads/
Allow /wp-admin/admin-ajax.php
Allow /wp-includes/*.css
Allow /wp-includes/*.js

agentic

Rule Path
Disallow /

ai article writer

Rule Path
Disallow /

ai chat

Rule Path
Disallow /

ai content detector

Rule Path
Disallow /

ai dungeon

Rule Path
Disallow /

ai search

Rule Path
Disallow /

ai seo crawler

Rule Path
Disallow /

ai writer

Rule Path
Disallow /

ai-proxy

Rule Path
Disallow /

ai21 labs

Rule Path
Disallow /

ai2bot

Rule Path
Disallow /

aibot

Rule Path
Disallow /

aihitbot

Rule Path
Disallow /

aimatrix

Rule Path
Disallow /

aisearchbot

Rule Path
Disallow /

ai training

Rule Path
Disallow /

aitraining

Rule Path
Disallow /

alexa

Rule Path
Disallow /

alpha ai

Rule Path
Disallow /

alphaai

Rule Path
Disallow /

amazon bedrock

Rule Path
Disallow /

amazon-kendra

Rule Path
Disallow /

amazon lex

Rule Path
Disallow /

amazon comprehend

Rule Path
Disallow /

amazon sagemaker

Rule Path
Disallow /

amazon silk

Rule Path
Disallow /

amazon textract

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

amelia

Rule Path
Disallow /

anderspinkbot

Rule Path
Disallow /

andibot

Rule Path
Disallow /

anthropic

Rule Path
Disallow /

anypicker

Rule Path
Disallow /

anyword

Rule Path
Disallow /

applebot

Rule Path
Disallow /

aria browse

Rule Path
Disallow /

articoolo

Rule Path
Disallow /

autoglm

Rule Path
Disallow /

automated writer

Rule Path
Disallow /

automl

Rule Path
Disallow /

awariorssbot

Rule Path
Disallow /

awariosmartbot

Rule Path
Disallow /

azure

Rule Path
Disallow /

babyagi

Rule Path
Disallow /

bardbot

Rule Path
Disallow /

brave leo

Rule Path
Disallow /

brightbot

Rule Path
Disallow /

bytedance

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

catboost

Rule Path
Disallow /

cc-crawler

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

chatbot

Rule Path
Disallow /

chatglm

Rule Path
Disallow /

chinchilla

Rule Path
Disallow /

claude

Rule Path
Disallow /

clearscope

Rule Path
Disallow /

cohere

Rule Path
Disallow /

common crawl

Rule Path
Disallow /

commoncrawl

Rule Path
Disallow /

content harmony

Rule Path
Disallow /

content king

Rule Path
Disallow /

content optimizer

Rule Path
Disallow /

content samurai

Rule Path
Disallow /

contentatscale

Rule Path
Disallow /

contentbot

Rule Path
Disallow /

contentedge

Rule Path
Disallow /

conversion ai

Rule Path
Disallow /

copilot

Rule Path
Disallow /

copyai

Rule Path
Disallow /

copymatic

Rule Path
Disallow /

copyscape

Rule Path
Disallow /

cotoyogi

Rule Path
Disallow /

crawlq ai

Rule Path
Disallow /

crawlspace

Rule Path
Disallow /

crew ai

Rule Path
Disallow /

crewai

Rule Path
Disallow /

dall-e

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

dataprovider

Rule Path
Disallow /

deepai

Rule Path
Disallow /

deepl

Rule Path
Disallow /

deepmind

Rule Path
Disallow /

deepseek

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

doubao ai

Rule Path
Disallow /

duckassistbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

facebookexternalhit

Rule Path
Disallow /

factset

Rule Path
Disallow /

falcon

Rule Path
Disallow /

firecrawl

Rule Path
Disallow /

flyriver

Rule Path
Disallow /

frase ai

Rule Path
Disallow /

friendlycrawler

Rule Path
Disallow /

gemini

Rule Path
Disallow /

gemma

Rule Path
Disallow /

genai

Rule Path
Disallow /

genspark

Rule Path
Disallow /

glm

Rule Path
Disallow /

goose

Rule Path
Disallow /

gpt

Rule Path
Disallow /

grammarly

Rule Path
Disallow /

grendizer

Rule Path
Disallow /

grok

Rule Path
Disallow /

gt bot

Rule Path
Disallow /

gtbot

Rule Path
Disallow /

hemingway editor

Rule Path
Disallow /

hugging face

Rule Path
Disallow /

hypotenuse ai

Rule Path
Disallow /

iaskspider

Rule Path
Disallow /

icc-crawler

Rule Path
Disallow /

imagegen

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

img2dataset

Rule Path
Disallow /

imgproxy

Rule Path
Disallow /

ink editor

Rule Path
Disallow /

inkforall

Rule Path
Disallow /

intelliseek

Rule Path
Disallow /

inferkit

Rule Path
Disallow /

isscyberriskcrawler

Rule Path
Disallow /

jasperai

Rule Path
Disallow /

kafkai

Rule Path
Disallow /

kangaroo

Rule Path
Disallow /

keyword density ai

Rule Path
Disallow /

knowledge

Rule Path
Disallow /

komobot

Rule Path
Disallow /

lensa

Rule Path
Disallow /

lightpanda

Rule Path
Disallow /

llama

Rule Path
Disallow /

llms

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

manus

Rule Path
Disallow /

marketmuse

Rule Path
Disallow /

meltwater

Rule Path
Disallow /

meta ai

Rule Path
Disallow /

meta-ai

Rule Path
Disallow /

meta-external

Rule Path
Disallow /

metaai

Rule Path
Disallow /

metatagbot

Rule Path
Disallow /

mistral

Rule Path
Disallow /

monica

Rule Path
Disallow /

narrative

Rule Path
Disallow /

neevabot

Rule Path
Disallow /

neural text

Rule Path
Disallow /

neuralseo

Rule Path
Disallow /

nova act

Rule Path
Disallow /

novaact

Rule Path
Disallow /

oai-searchbot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

open ai

Rule Path
Disallow /

openai

Rule Path
Disallow /

openbot

Rule Path
Disallow /

opentext ai

Rule Path
Disallow /

operator

Rule Path
Disallow /

outwrite

Rule Path
Disallow /

page analyzer ai

Rule Path
Disallow /

pangubot

Rule Path
Disallow /

paperlibot

Rule Path
Disallow /

paraphraser.io

Rule Path
Disallow /

perplexity

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

phind

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

prowritingaid

Rule Path
Disallow /

puppeteer

Rule Path
Disallow /

qualified

Rule Path
Disallow /

quillbot

Rule Path
Disallow /

qwen

Rule Path
Disallow /

robotspider

Rule Path
Disallow /

rytr

Rule Path
Disallow /

saplingai

Rule Path
Disallow /

scalenut

Rule Path
Disallow /

scraper

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

scriptbook

Rule Path
Disallow /

seo content machine

Rule Path
Disallow /

seo robot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

sentibot

Rule Path
Disallow /

sidetrade

Rule Path
Disallow /

simplified ai

Rule Path
Disallow /

sitefinity

Rule Path
Disallow /

skydancer

Rule Path
Disallow /

slickwrite

Rule Path
Disallow /

sonic

Rule Path
Disallow /

spin rewriter

Rule Path
Disallow /

spinbot

Rule Path
Disallow /

stability

Rule Path
Disallow /

stablediffusionbot

Rule Path
Disallow /

sudowrite

Rule Path
Disallow /

super agent

Rule Path
Disallow /

surfer ai

Rule Path
Disallow /

text blaze

Rule Path
Disallow /

textcortex

Rule Path
Disallow /

the knowledge ai

Rule Path
Disallow /

tiktokspider

Rule Path
Disallow /

timpibot

Rule Path
Disallow /

traefik

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

vidnami ai

Rule Path
Disallow /

webzio

Rule Path
Disallow /

whisper

Rule Path
Disallow /

wordai

Rule Path
Disallow /

wordtune

Rule Path
Disallow /

wormsgtp

Rule Path
Disallow /

wpbot

Rule Path
Disallow /

writecream

Rule Path
Disallow /

writerzen

Rule Path
Disallow /

writescope

Rule Path
Disallow /

writesonic

Rule Path
Disallow /

xai

Rule Path
Disallow /

xbot

Rule Path
Disallow /

yaml

Rule Path
Disallow /

youbot

Rule Path
Disallow /

zero gtp

Rule Path
Disallow /

zerochat

Rule Path
Disallow /

zhipu

Rule Path
Disallow /

zimm

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://mywebmaestro.com/sitemap_index.xml
Back to top

Comments

  • General rules for all user agents
  • Allow Important Assets
  • Yoast SEO Sitemap Location
  • Ultimate AI Block List v1.5 20250603
  • https://perishablepress.com/ultimate-ai-block-list/
Back to top

Warnings

  • 2 invalid lines.
Back to top