Pull requests / #3077
#3077 docs: ROCm 6.3.3 / gfx906 report — 2x MI50, system ROCm, build + run + 12 h split-config soak
open · @ken2425pm · 0 コメント · GitHub で見る
BenchmarksSetup & installServer & APIMulti-GPUAMD / HIPNVIDIA / CUDAModels & quantsDocumentation
本文
Hardware the maintainers do not have, per docs/COMMUNITY_BENCHMARKS.md: two Instinct MI50 32 GB (gfx906) on a dual Xeon E5-2630 v3 box (16C/32T, 256 GB DDR4), system ROCm 6.3.3 at `/opt/rocm-6.3.3` (7.x dropped gfx906 bitcode, so 6.x is the newest line that still ships the libraries). **What it adds** - `docs/ROCm_6_3_3.md` (new, ~310 lines): the full report — compat layer, per-file port/fix list, build verification, runtime validation (warm decode 21-24 tok/s on the final config vs 11.6 tok/s on the earlier verified tree — not a same-request A/B), a CPU-affinity sweep, serve-mode A/Bs (spec 4 + min-p 0.5 kept, kv-resident 32768 kept, VRAM reserve 1024 → 768 with re-test pending), dual-GPU layer-split (~+20% decode vs one clean single GPU) and thread-count A/Bs, known limitations, and a verification checklist. - `docs/AMD_HIP.md`: a "With a system ROCm 6.x" subsection in the gfx906 section — the exact `cmake` invocation and what it verifies. **Answers TEST_REQUESTS.md**: the "long soak (an hour or more) on a split setup" row — 12 hours of continuous resident-serve use of the 256K-context / 2-batch-slot split config: no crashes, no restarts, no paging warnings, no VRAM or expert-cache degradation observed. **Style**: plain words, every number says what it was measured on. Setup-side gfx906 recognition is *not* claimed here — it is proposed in a separate change, and the doc says so. **Not tested here**: CUDA and SYCL builds (no CUDA/oneAPI toolchain on this box), a `-Werror` build (measured and documented in §8.1 as not taken), deep-context KV-streaming cost and 64K+ prefill timing (left open in the checklist). gfx900/MI-25 is out of scope for this report.
関連リンク
インストール・モデル・リリースへの站内リンク。