#4154 [Docs] ROCm 6.3.3 / gfx906 report — 2x MI50, system ROCm, build + run + 12 h split-config soak

open · @ken2425pm · 0 comentarios · En GitHub

Server & APIMulti-GPUAMD / HIPNVIDIA / CUDAModels & quantsDocumentation

Descripción

Replaces the same-numbered closed PR (0.2.0-beta tracker reset; its /reopen check passed but GitHub would not reopen the closed PR after the branch update - new PR per the check bot's guidance).

Problem and result: a gfx906 (2x Instinct MI50 32 GB) build-and-run report on a **system** ROCm 6.3.3 — hardware the maintainers do not have — plus a "With a system ROCm 6.3.3, no container" build path in `docs/AMD_HIP.md`. Answers the `TEST_REQUESTS.md` soak row: 12 hours of continuous resident-serve on the 256K-context / 2-batch-slot split config — no crashes, no restarts, no paging warnings, no VRAM or expert-cache degradation observed.

Related discussion: #2883 (community gfx906 validation), #3172 (the current 0.2.x compile gaps — the report's 0.2.x status note points there), the closed #3077 of the same content (0.2.0-beta tracker reset).

Approach: `docs/ROCm_6_3_3.md` (new) is the full report — compat layer, per-file port/fix list, build verification, runtime numbers, serve-mode A/Bs, dual-GPU layer split (~+20% decode vs one clean single GPU), known limits, and a verification checklist. The `AMD_HIP.md` addition documents the no-container system-ROCm build (cmake fetches the pinned ggml itself; no `STRATA_GGML_DIR` needed).

**Power note (added on reopen):** the GPU power limit was set to **125 W per card** (both MI50s) during every performance capture below — the numbers are lower bounds. A full-power re-capture is planned and will be appended.

Validation:
- Passed: the `AMD_HIP.md` build command, run today on this checkout (v0.2.6): configure runs, and with the two compile fixes tracked in the [Engine] alignas PR and #3172, `ninja -C build-906 strata` links (35.7 MB). The report's runtime numbers were measured on the 0.1.45 base (`477ac13`) on this box; the header and §2 state that.
- Failed: none observed.
- Not run: a same-day 0.2.6 performance re-capture (planned — with and without the power cap); CUDA and SYCL builds (no toolchains here); a `-Werror` build (measured as not taken, §8.1).

Limits: the measurements are 0.1.45-base and power-capped (stated in the header); deep-context KV-streaming cost and 64K+ prefill timing remain open in the checklist.

En el sitio

Enlaces a install, modelos, releases.