Log In Sign Up

tsc.openle.com
robots.txt

Robots Exclusion Standard data for tsc.openle.com

Resource Scan

Scan Details

Site Domain tsc.openle.com
Base Domain openle.com
Scan Status Ok
Last Scan2025-08-26T16:56:26+00:00
Next Scan 2025-09-02T16:56:26+00:00

Last Scan

Scanned2025-08-26T16:56:26+00:00
URL https://tsc.openle.com/robots.txt
Domain IPs 121.196.244.226
Response IP 121.196.244.226
Found Yes
Hash 52093f3d10c248e1cd7462da14fb805f6e6e78adfa528130f7def64a931b1d5b
SimHash ec04b66b9d50

Groups

*

Rule Path
Disallow /
Allow /$
Allow /ads.txt
Allow /.well-known/static/images/core/
Allow /.well-known/static/images/more/
Allow /main/core/
Disallow /main/core/auth/accounts/
Allow /main/more/
Disallow /main/apis/core/
Disallow /main/apis/more/
Disallow /main/temp/
Allow /app-ads.txt
Allow /.well-known/general/privacy/
Allow /main/go/v1/our/
Back to top

Comments

  • Latest version - https://nav.congci.com/main/home/topics/work-init/#robots.txt
  • RFC 9309 - https://www.robotstxt.org/
  • Google 缓存 robots.txt 约 24 小时；HTTP 5xx 和 429 均解读为网站完å
  • 可在任意位置指定多个sitemap文件。
  • Sitemap: https://example.com/.well-known/static/sitemap.xml
  • Sitemap: https://example.com/.well-known/static/sitemapindex.xml
  • 约定：固定路径 /main/core/topics/ 之后的第一层总是带/斜杠，å
  • https://example.com/main/core/about/
  • https://example.com/main/core/mobile/
  • https://example.com/main/core/privacy/
  • https://example.com/main/core/trending/
  • https://example.com/main/core/products/
  • https://example.com/main/core/products/come-in-handy
  • https://example.com/main/core/products/3rd/tile-matching
  • https://example.com/main/core/topics/
  • https://example.com/main/core/topics/ai/
  • https://example.com/main/core/topics/ai/chat
  • https://example.com/main/core/topics/ai/chat/tool
  • https://example.com/main/core/auth/login/
  • https://example.com/main/core/auth/signup/
  • https://example.com/main/core/auth/accounts/ - 禁止抓取；欢迎页结尾带斜杠可避å
  • https://example.com/main/apis/core/auth/callback/ - 禁止抓取；
  • 白名单机制；“/$”表示只匹é
  • 图片
  • 约定path：
  • https://example.com/.well-known/static/images/core/logo.svg
  • 正式对外核心网页(或用web)
  • 正式对外非核心网页(或用any)
  • 约定path：
  • https://example.com/main/more/seo/sitemap.xml
  • 正式对外核心应用接口(或用use) - /main/apis/core/pv
  • 正式对外非核心应用接口(或用any) - /main/apis/more/x/execute
  • 不对外、任意使用的网址前缀(ban、not、for) - /main/temp/x
  • 特例
  • ä»
  • Allow: /main/go/v1/page/
Back to top

Warnings

  • 4 invalid lines.
  • `¨ç¦æ­¢è®¿é—®â€œdisallow` is not a known field.
  • `部用？ https` is not a known field.
Back to top