trex.xyz
robots.txt

Robots Exclusion Standard data for trex.xyz

Resource Scan

Scan Details

Site Domain trex.xyz
Base Domain trex.xyz
Scan Status Ok
Last Scan2025-10-15T00:56:39+00:00
Next Scan 2025-11-14T00:56:39+00:00

Last Scan

Scanned2025-10-15T00:56:39+00:00
URL https://trex.xyz/robots.txt
Redirect https://www.trex.xyz/robots.txt
Redirect Domain www.trex.xyz
Redirect Base trex.xyz
Domain IPs 216.150.1.1
Redirect IPs 216.150.1.65, 216.150.16.65
Response IP 216.150.16.1
Found Yes
Hash d03a1d75a841370531fea8ce39516b29ae92a7cc37f93c7fab14719865aba48b
SimHash 481cf2c1cb22

Groups

*

Rule Path
Allow /
Allow /api/og
Disallow /api/
Disallow /.well-known/
Disallow /_next/static/

twitterbot

Rule Path
Allow /api/og

facebookexternalhit

Rule Path
Allow /api/og

Other Records

Field Value
sitemap https://www.trex.xyz/sitemap.xml
sitemap https://trex.xyz/sitemap.xml

Comments

  • 禁止抓取某些特定路径
  • 允许 Twitter 和其他社交媒体抓取 OG 图片
  • 站点地图位置