Log In Sign Up

detailedmanual.net
robots.txt

Robots Exclusion Standard data for detailedmanual.net

Resource Scan

Scan Details

Site Domain detailedmanual.net
Base Domain detailedmanual.net
Scan Status Ok
Last Scan2025-10-05T06:49:36+00:00
Next Scan 2025-10-12T06:49:36+00:00

Last Scan

Scanned2025-10-05T06:49:36+00:00
URL https://detailedmanual.net/robots.txt
Domain IPs 104.21.49.30, 172.67.140.130, 2606:4700:3033::6815:311e, 2606:4700:3033::ac43:8c82
Response IP 172.67.140.130
Found Yes
Hash 856de4605319b0826e1b1854d3157384c074558072accdb04e6840b159ff9424
SimHash 76e7d0089d44

Groups

*

Rule Path
Allow /

amazonbot

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Disallow /wp-includes/
Allow /wp-includes/js/
Allow /wp-includes/images/
Disallow /trackback/
Disallow /wp-login.php
Disallow /wp-register.php

gptbot

Product Comment
gptbot OpenAI: ChatGPT 모델 학습용 크롤러
Rule Path
Disallow /

chatgpt-user

Product Comment
chatgpt-user OpenAI: ChatGPT 플러그인/Action 등 사용자 요청 기반 브라우징
Rule Path
Disallow /

chatgpt-user/2.0

Product Comment
chatgpt-user/2.0 OpenAI: ChatGPT 브라우징 최신 버전 추정
Rule Path
Disallow /

oai-searchbot/1.0

Product Comment
oai-searchbot/1.0 OpenAI: 검색 관련 기능 봇 추정
Rule Path
Disallow /

google-extended

Product Comment
google-extended Google: Vertex AI 등 AI 모델 학습 데이터 수집 (Google 검색 색인과는 무관)
Rule Path
Disallow /

claudebot

Product Comment
claudebot Anthropic: Claude AI 모델 관련 봇
Rule Path
Disallow /

claude-web

Product Comment
claude-web Anthropic: Claude 웹 브라우징 기능 관련 봇 (추정)
Rule Path
Disallow /

anthropic-ai

Product Comment
anthropic-ai Anthropic: 일반적인 봇 식별자
Rule Path
Disallow /

perplexitybot

Product Comment
perplexitybot Perplexity AI: AI 검색엔진 크롤러
Rule Path
Disallow /

perplexity-user/1.0

Product Comment
perplexity-user/1.0 Perplexity AI: 사용자 요청 기반 브라우징 (추정)
Rule Path
Disallow /

cohere-ai

Product Comment
cohere-ai Cohere: AI 모델 관련 봇
Rule Path
Disallow /

cohere-training-data-crawler

Product Comment
cohere-training-data-crawler Cohere: 명시적인 학습 데이터 수집 크롤러
Rule Path
Disallow /

wrtnbot

Rule Path
Disallow /

mistralai-user/1.0

Product Comment
mistralai-user/1.0 Mistral AI: 사용자 요청 기반 브라우징 (추정)
Rule Path
Disallow /

youbot

Product Comment
youbot You.com: AI 기반 검색엔진 크롤러
Rule Path
Disallow /

deepseek-crawler

Product Comment
deepseek-crawler DeepSeek AI: AI 모델 관련 크롤러
Rule Path
Disallow /

brightbot/1.0

Product Comment
brightbot/1.0 BrightEdge: SEO 및 AI 분석 플랫폼 크롤러
Rule Path
Disallow /

ai2bot/1.0

Product Comment
ai2bot/1.0 Allen Institute for AI: Semantic Scholar 학술 검색 관련 봇
Rule Path
Disallow /

ccbot

Product Comment
ccbot Common Crawl: 비영리 웹 아카이빙 및 AI 학습 데이터셋 구축
Rule Path
Disallow /

bytespider

Product Comment
bytespider ByteDance: TikTok 모회사, 데이터 수집 및 AI 학습용
Rule Path
Disallow /

metabot

Product Comment
metabot Meta: Facebook, Instagram 등 메타 플랫폼용 봇
Rule Path
Disallow /

meta-externalagent

Product Comment
meta-externalagent Meta: 외부 링크 처리 등 관련 봇 (추정)
Rule Path
Disallow /

meta-externalfetcher

Product Comment
meta-externalfetcher Meta: 외부 데이터 가져오기 관련 봇 (추정)
Rule Path
Disallow /

facebookbot

Product Comment
facebookbot Meta: 구 버전 또는 특정 목적의 Facebook 봇
Rule Path
Disallow /

applebot

Product Comment
applebot Apple: Siri, Spotlight 제안 등 Apple 서비스용 데이터 수집
Rule Path
Disallow /

omgilibot

Product Comment
omgilibot 웹 콘텐츠 집계 서비스
Rule Path
Disallow /

omgili

Product Comment
omgili Omgili의 다른 User-agent
Rule Path
Disallow /

diffbot

Product Comment
diffbot 웹 페이지 구조화 및 데이터 추출 서비스
Rule Path
Disallow /

dataforseobot

Product Comment
dataforseobot SEO 데이터 제공 서비스 크롤러
Rule Path
Disallow /

blexbot

Product Comment
blexbot WebMeUp: 웹 분석 및 데이터 수집 서비스
Rule Path
Disallow /

ahrefsbot

Product Comment
ahrefsbot Ahrefs: SEO 분석 도구 크롤러
Rule Path
Disallow /

ahrefssiteaudit

Product Comment
ahrefssiteaudit Ahrefs: 사이트 감사 도구용 크롤러
Rule Path
Disallow /

semrushbot

Product Comment
semrushbot Semrush: SEO 분석 도구 크롤러 (다양한 변형 존재)
Rule Path
Disallow /

semrushbot-sa

Product Comment
semrushbot-sa Semrush: 사이트 감사 도구용 크롤러
Rule Path
Disallow /

semrushbot-ba

Product Comment
semrushbot-ba Semrush: 백링크 감사 도구용 크롤러
Rule Path
Disallow /

semrushbot-seo

Product Comment
semrushbot-seo Semrush: 기타 SEO 분석 관련 크롤러
Rule Path
Disallow /

mj12bot

Product Comment
mj12bot Majestic: 백링크 분석 전문 도구 크롤러
Rule Path
Disallow /

dotbot

Product Comment
dotbot Moz: SEO 분석 도구 크롤러
Rule Path
Disallow /

cotoyogi

Product Comment
cotoyogi 상세 불명, 공격적 크롤링 사례 보고됨
Rule Path
Disallow /

crawlspace

Product Comment
crawlspace 상세 불명 크롤러
Rule Path
Disallow /

firecrawlagent

Product Comment
firecrawlagent AI 애플리케이션용 웹사이트 데이터 변환 서비스 추정
Rule Path
Disallow /

friendlycrawler

Product Comment
friendlycrawler 이름과 달리 공격적 크롤링 사례 보고됨
Rule Path
Disallow /

factset_spyderbot

Product Comment
factset_spyderbot FactSet: 금융 데이터 수집 봇
Rule Path
Disallow /

petalbot

Product Comment
petalbot Huawei: 화웨이 검색 서비스(Petal Search) 크롤러
Rule Path
Disallow /

operator

Product Comment
operator 상세 불명 크롤러
Rule Path
Disallow /

pangubot

Product Comment
pangubot 상세 불명 크롤러
Rule Path
Disallow /

novaact

Product Comment
novaact 상세 불명 크롤러
Rule Path
Disallow /

velenpublicwebcrawler

Product Comment
velenpublicwebcrawler 상세 불명 크롤러
Rule Path
Disallow /

kangaroo bot

Product Comment
kangaroo bot 상세 불명 크롤러
Rule Path
Disallow /

webzio-extended

Product Comment
webzio-extended AI용 웹 데이터 피드 서비스 추정
Rule Path
Disallow /

amazonbot/0.1

Product Comment
amazonbot/0.1 Amazon: 상품 정보, Alexa 등 아마존 서비스 관련 봇
Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://detailedmanual.net/sitemap_index.xml
Back to top

Comments

  • As a condition of accessing this website, you agree to abide by the following
  • content signals:
  • (a) If a content-signal = yes, you may collect content for the corresponding
  • use.
  • (b) If a content-signal = no, you may not collect content for the
  • corresponding use.
  • (c) If the website operator does not include a content signal for a
  • corresponding use, the website operator neither grants nor restricts
  • permission via content signal with respect to the corresponding use.
  • The content signals and their meanings are:
  • search: building a search index and providing search results (e.g., returning
  • hyperlinks and short excerpts from your website's contents). Search does not
  • include providing AI-generated search summaries.
  • ai-input: inputting content into one or more AI models (e.g., retrieval
  • augmented generation, grounding, or other real-time taking of content for
  • generative AI search answers).
  • ai-train: training or fine-tuning AI models.
  • ANY RESTRICTIONS EXPRESSED VIA CONTENT SIGNALS ARE EXPRESS RESERVATIONS OF
  • RIGHTS UNDER ARTICLE 4 OF THE EUROPEAN UNION DIRECTIVE 2019/790 ON COPYRIGHT
  • AND RELATED RIGHTS IN THE DIGITAL SINGLE MARKET.
  • BEGIN Cloudflare Managed content
  • END Cloudflare Managed Content
  • This virtual robots.txt file was created by the Virtual Robots.txt WordPress plugin: https://www.wordpress.org/plugins/pc-robotstxt/
  • XML Sitemap
  • ===================================================================
  • 모든 로봇에 대한 일반 규칙 (검색 엔진 포함)
  • 일반 검색 엔진은 아래 명시적으로 차단된 봇을 제외하고 접근이 허용됩니다.
  • ===================================================================
  • Disallow: /readme.html # 선택 사항: 워드프레스 버전 정보 등 노출 방지
  • Disallow: /license.txt # 선택 사항: 라이선스 정보 노출 방지
  • ===================================================================
  • AI · 대규모 데이터 수집 · 주요 SEO 분석 크롤러 차단 목록
  • ===================================================================
  • --- OpenAI / ChatGPT 관련 ---
  • --- Google AI / Vertex AI ---
  • --- Anthropic / Claude ---
  • --- Perplexity AI ---
  • --- Cohere AI ---
  • --- Wrtn AI ---
  • --- Mistral AI ---
  • --- You.com ---
  • --- DeepSeek AI ---
  • --- BrightEdge (SEO+AI Platform) ---
  • --- Allen Institute for AI (Semantic Scholar) ---
  • ===================================================================
  • 플랫폼·국가 기반 대규모 수집 크롤러
  • ===================================================================
  • ===================================================================
  • 웹 데이터 수집 · 집계 · 분석 서비스 크롤러
  • ===================================================================
  • ===================================================================
  • 주요 SEO 분석 도구 크롤러 (리소스 사용량 높음)
  • ===================================================================
  • ※ 사용 중인 도구 봇은 주석 처리('# ')하거나 해당 블록을 삭제하세요.
  • ===================================================================
  • 기타 잠재적 데이터 수집·과도한 크롤링 봇
  • ===================================================================
Back to top

Warnings

  • `content-signal` is not a known field.
Back to top