bocm.no
robots.txt

Robots Exclusion Standard data for bocm.no

Resource Scan

Scan Details

Site Domain bocm.no
Base Domain bocm.no
Scan Status Ok
Last Scan2025-10-21T09:10:27+00:00
Next Scan 2025-11-04T09:10:27+00:00

Last Scan

Scanned2025-10-21T09:10:27+00:00
URL https://bocm.no/robots.txt
Domain IPs 95.217.68.68
Response IP 95.217.68.68
Found Yes
Hash c98ab571f3668caac48713ea807b6a7e8c2991403dc363f553425c11229ea9c3
SimHash 4816d854f9d0

Groups

semrushbot
ahrefsbot
serpstatbot
dataforseobot
mj12bot
sentibot
yandexbot
petalbot
neticle
buck
cookieinformationscanner
trendictionbot
velenpublicwebcrawler
gptbot
oai-searchbot
openai-searchbot
chatgpt-user
ccbot
claudebot
anthropic-ai
claude-web
perplexitybot
perplexity-user
google-extended
googlebot-extended
meta-externalagent
meta-externalfetcher
amazonbot
applebot
applebot-extended
bytespider
cohere-ai
ai2bot
diffbot
youbot
mistralai-user
duckassistbot
timpibot
other

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

oai-searchbot

Rule Path
Disallow /

openai-searchbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

perplexity-user

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

googlebot-extended

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

meta-externalfetcher

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

applebot

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

ai2bot

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

youbot

Rule Path
Disallow /

mistralai-user

Rule Path
Disallow /

duckassistbot

Rule Path
Disallow /

timpibot

Rule Path
Disallow /

*

Rule Path
Disallow /webdesk/
Disallow /search
Disallow /index.php*%26do%3Dsok*
Disallow /api/
Disallow /conf/
Disallow /src/lang/
Disallow /src/nocache/
Disallow /*download.php
Disallow /src/download.php
Disallow /dashboard.php
Disallow /buyandread/
Disallow /minside/tipsenvenn/
Disallow /pageflow/
Disallow /minside/brukere/
Disallow /src/lib/blackhole/
Disallow /*?*ac=*
Disallow /*/ac/*

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://bocm.no/sitemap.xml

Comments

  • AI Crawlers - Please respect rate limits (2 requests per minute)