贡献 / #1969

#1969 Community rtx pro 4500 blackwell 600k

open · @HONGJUNZZY · 0 评论 · 去 GitHub 看

BenchmarksAMD / HIPNVIDIA / CUDAModels & quantsLinux

说明

First RTX PRO 4500 Blackwell (sm_120) entry in the community list.

Machine: AMD EPYC 7A23 48-core, 123 GiB RAM, one RTX PRO 4500 32 GB, Linux, source build of v0.1.43 (7695859a).

Production config measured as-is: Unsloth UD-IQ4_XS, 600,000-token context with yarn ×2.29, `--kv int8`,
`--kv-resident 300000` (50 % of the KV in VRAM, the rest pinned in RAM), expert cache auto (8,052 slots).

Built-in suite `tools/strata_bench.py --long`, 3 runs per test, medians (engine's own timings):

- short chat: decode 82.6 tok/s (55.1–120.1), TTFT 0.33 s
- prompt: 1,718 (4K) / 3,057 (16K) / 3,242 (32K) tok/s
- agent session (6 turns to 14.5K tokens): decode 97.9 tok/s, TTFT 2.13 s/turn,
  conversation cache hit 68.8 %, only 4 tokens re-read per session

Folder generated by the suite itself; secrets/paths sanitized (verified before submitting).

本站相关内容

相关页面的快捷入口。