Pull requests / #1969
#1969 Community rtx pro 4500 blackwell 600k
open · @HONGJUNZZY · 0 Kommentare · Auf GitHub
BenchmarksAMD / HIPNVIDIA / CUDAModels & quantsLinux
Beschreibung
First RTX PRO 4500 Blackwell (sm_120) entry in the community list. Machine: AMD EPYC 7A23 48-core, 123 GiB RAM, one RTX PRO 4500 32 GB, Linux, source build of v0.1.43 (7695859a). Production config measured as-is: Unsloth UD-IQ4_XS, 600,000-token context with yarn ×2.29, `--kv int8`, `--kv-resident 300000` (50 % of the KV in VRAM, the rest pinned in RAM), expert cache auto (8,052 slots). Built-in suite `tools/strata_bench.py --long`, 3 runs per test, medians (engine's own timings): - short chat: decode 82.6 tok/s (55.1–120.1), TTFT 0.33 s - prompt: 1,718 (4K) / 3,057 (16K) / 3,242 (32K) tok/s - agent session (6 turns to 14.5K tokens): decode 97.9 tok/s, TTFT 2.13 s/turn, conversation cache hit 68.8 %, only 4 tokens re-read per session Folder generated by the suite itself; secrets/paths sanitized (verified before submitting).
Mehr auf der Site
Links zu Install, Modellen, Releases.