Log In Sign Up

knowprose.com
robots.txt

Robots Exclusion Standard data for knowprose.com

Resource Scan

Scan Details

Site Domain knowprose.com
Base Domain knowprose.com
Scan Status Ok
Last Scan2026-01-01T22:11:52+00:00
Next Scan 2026-01-02T22:11:52+00:00

Last Scan

Scanned2026-01-01T22:11:52+00:00
URL https://knowprose.com/robots.txt
Domain IPs 191.101.228.55, 2a02:4780:38:5a0f:e517:fb1b:9609:989e, 2a02:4780:39:7438:efe4:58c0:d924:4d69, 93.127.196.51
Response IP 77.37.66.110
Found Yes
Hash 39f2e77808a33b3f5e30b4a48ca0a69a116f004504f2bc6fe525a7261f6bd764
SimHash cd1d4b9302c2

Groups

googlebot

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

googlebot-news

Rule Path
Allow /

google-pagespeed

Rule Path
Allow /

google-site-verification

Rule Path
Allow /

lighthouse

Rule Path
Allow /

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php

addsearchbot

Rule Path
Disallow /

ai2bot

Rule Path
Disallow /
Disallow /

ai2bot-dolma

Rule Path
Disallow /
Disallow /

aihitbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /
Disallow /

applebot-extended

Rule Path
Disallow /
Disallow /

anthropic-ai

Rule Path
Disallow /
Disallow /

bedrockbot

Rule Path
Disallow /

bigsur.ai

Rule Path
Disallow /

brightbot 1.0

Rule Path
Disallow /

bytespider

Rule Path
Disallow /
Disallow /

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /
Disallow /

claudebot

Rule Path
Disallow /
Disallow /

claude-user

Rule Path
Disallow /
Disallow /

claude-searchbot

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /
Disallow /

cohere-training-data-crawler

Rule Path
Disallow /
Disallow /

cotoyogi

Rule Path
Disallow /

crawlspace

Rule Path
Disallow /

deepseekbot

Rule Path
Disallow /

diffbot

Rule Path
Disallow /
Disallow /

echoboxbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /
Disallow /

factset_spyderbot

Rule Path
Disallow /

firecrawlagent

Rule Path
Disallow /

friendlycrawler

Rule Path
Disallow /

gptbot

Rule Path
Disallow /
Disallow /

googleagent-mariner

Rule Path
Disallow /

gemini-deep-research

Rule Path
Disallow /

google-extended

Rule Path
Disallow /
Disallow /

imagesiftbot

Rule Path
Disallow /
Disallow /

kangaroo bot

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

meta-externalfetcher

Rule Path
Disallow /

oai-searchbot

Rule Path
Disallow /
Disallow /

omgili

Rule Path
Disallow /
Disallow /

omgilibot

Rule Path
Disallow /
Disallow /

pangubot

Rule Path
Disallow /
Disallow /

panscient

Rule Path
Disallow /

panscient.com

Rule Path
Disallow /

petalbot

Rule Path
Disallow /
Disallow /

perplexitybot

Rule Path
Disallow /
Disallow /

perplexityuser

Rule Path
Disallow /

poseidon research crawler

Rule Path
Disallow /

sbintuitionsbot

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-ocob

Rule Path
Disallow /

semrushbot-ft

Rule Path
Disallow /

sentibot

Rule Path
Disallow /

sentibot

Rule Path
Disallow /
Disallow /

terracotta

Rule Path
Disallow /

thinkbot

Rule Path
Disallow /

timpibot

Rule Path
Disallow /
Disallow /

turnitinbot

Rule Path
Disallow /
Disallow /

yak

Rule Path
Disallow /

yandexadditional

Rule Path
Disallow /

yandexadditionalbot

Rule Path
Disallow /

youbot

Rule Path
Disallow /
Disallow /

webzio

Rule Path
Disallow /
Disallow /

webzio-extended

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

bardbot

Rule Path
Disallow /

komobot

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

meta-externalfetcher

Rule Path
Disallow /

webzio-extended

Rule Path
Disallow /

stablediffusionbot

Rule Path
Disallow /

grok

Rule Path
Disallow /

grokai

Rule Path
Disallow /

xai

Rule Path
Disallow /

xbot

Rule Path
Disallow /

duckassistbot

Rule Path
Disallow /

img2dataset

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

openaicontentcrawler

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

neevabot

Rule Path
Disallow /

aimatrixcrawler

Rule Path
Disallow /

amazon-ai

Rule Path
Disallow /

anthropicbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

googleother

Rule Path
Disallow /

chatglm-spider

Rule Path
Disallow /

sogou

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

erniebot

Rule Path
Disallow /

deepseekbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://knowprose.com/sitemap.xml
sitemap https://knowprose.com/sitemap.rss
Back to top

Warnings

  • 2 invalid lines.
Back to top