fated.link
robots.txt

Robots Exclusion Standard data for fated.link

Resource Scan

Scan Details

Site Domain fated.link
Base Domain fated.link
Scan Status Ok
Last Scan2025-10-19T01:03:20+00:00
Next Scan 2025-10-26T01:03:20+00:00

Last Scan

Scanned2025-10-19T01:03:20+00:00
URL https://fated.link/robots.txt
Domain IPs 3.34.66.169
Response IP 3.34.66.169
Found Yes
Hash fc05fd46cb7652cf7dce5f4704ce7f26d8b266fe0787e78dba7033b44f56263a
SimHash 49161cc79791

Groups

*

Rule Path
Allow /
Disallow /privacy_*.html
Disallow /terms_*.html

Other Records

Field Value
crawl-delay 1

googlebot

Rule Path
Allow /
Disallow /privacy_*.html
Disallow /terms_*.html

Other Records

Field Value
crawl-delay 1

bingbot

Rule Path
Allow /
Disallow /privacy_*.html
Disallow /terms_*.html

Other Records

Field Value
crawl-delay 1

naverbot

Rule Path
Allow /
Disallow /privacy_*.html
Disallow /terms_*.html

Other Records

Field Value
crawl-delay 1

Other Records

Field Value
sitemap https://fated.link/sitemap.xml

Comments

  • 사이트맵 위치
  • 크롤링 속도 조절 (1초당 1요청)
  • 주요 검색엔진별 설정