mitunolens.com
robots.txt

Robots Exclusion Standard data for mitunolens.com

Resource Scan

Scan Details

Site Domain mitunolens.com
Base Domain mitunolens.com
Scan Status Ok
Last Scan2026-02-11T01:58:17+00:00
Next Scan 2026-03-13T01:58:17+00:00

Last Scan

Scanned2026-02-11T01:58:17+00:00
URL https://mitunolens.com/robots.txt
Domain IPs 104.26.12.157, 104.26.13.157, 172.67.72.168, 2606:4700:20::681a:c9d, 2606:4700:20::681a:d9d, 2606:4700:20::ac43:48a8
Response IP 172.67.72.168
Found Yes
Hash f51d19a3ee5490f02b70379c88d05bf1e12cdb047f39e804b0cd6187b3259db6
SimHash ccd4c6a2a2d3

Groups

naverbot
yeti

Rule Path
Disallow /

*

Rule Path
Disallow /admin/
Disallow /home/product/review_write.php*
Disallow /home/product/qna_write.php*
Disallow /mobile/product_review_list.php*
Disallow /Mobile/product_review_list.php*
Disallow /mobile/product_qna_list.php*
Disallow /mobile/qnaform.php*
Disallow /board/board.html?*mid=review*

Other Records

Field Value
crawl-delay 10

Comments

  • 1. 네이버 봇 차단 (고객사 요청에 따라)
  • 2. 그 외 모든 봇에 대한 설정
  • 빙(Bing), 얀덱스 등 Crawl-delay를 지원하는 봇에게 10초 딜레이 요청
  • 중요하지 않거나 부하가 심한 페이지 차단