hlyun.org
robots.txt

Robots Exclusion Standard data for hlyun.org

Resource Scan

Scan Details

Site Domain hlyun.org
Base Domain hlyun.org
Scan Status Ok
Last Scan2025-10-08T20:03:17+00:00
Next Scan 2025-10-15T20:03:17+00:00

Last Scan

Scanned2025-10-08T20:03:17+00:00
URL https://hlyun.org/robots.txt
Redirect https://houlang.cloud/robots.txt
Redirect Domain houlang.cloud
Redirect Base houlang.cloud
Domain IPs 104.21.27.146, 172.67.142.239, 2606:4700:3034::6815:1b92, 2606:4700:3036::ac43:8eef
Redirect IPs 43.174.14.129
Response IP 43.174.14.129
Found Yes
Hash 8965cd8e6632bfd3fbb573e7a0fb1d802e9908ede554b08a7acb4257898c5a13
SimHash 411557448d11

Groups

*

Rule Path
Allow /
Disallow /.vitepress/
Disallow /node_modules/
Disallow /.git/
Disallow /.DS_Store
Disallow /*.json$
Disallow /*.lock$
Disallow /dist/
Disallow /.trae/
Allow /zh-CN/
Allow /zh-Hant/
Allow /en/
Allow /logo.svg
Allow /favicon.ico
Allow /*.webp$
Allow /*.jpg$
Allow /*.png$

googlebot

Rule Path
Allow /

bingbot

Rule Path
Allow /

baiduspider

Rule Path
Allow /

Other Records

Field Value
sitemap https://houlang.cloud/sitemap.xml

Comments

  • 指向sitemap
  • 禁止爬取临时文件和缓存文件
  • 禁止爬取开发和构建相关文件
  • 允许爬取所有语言版本
  • 允许爬取重要资源
  • 特定搜索引擎优化