#4766 [Engine] gfx906: build dflash2 for Radeon VII
open · @nfriedly · 0 コメント · GitHub で見る
BenchmarksMulti-GPUAMD / HIPNVIDIA / CUDAModels & quantsDocumentationWindows
本文
This is an extended version of #3986 (2-line change instead of 1-line, lol), related to #4156. It was necessary to build for my dual-Radeon VII system, otherwise it was taking a CUDA branch and failing.
(I believe that just setting STRATA_USE_HIP=1 would turn on other things that the Radeon VII doesn't support.)
agent-written details below.
Edit: I just noticed the agent referenced several log files. They're probably still on my system, so I can attach them here if requested. Also, this was done with the Strata v0.2.8 codebase, but I believe it applies to 0.2.9 as well.
---
## Problem and result
I was unable to build the engine for a Radeon VII before this change: `-DSTRATA_HIP_GFX906=ON` stopped in
`src/core/dflash2.cpp` with
```
src/core/dflash2.cpp:205:5: error: use of undeclared identifier 'cudaDeviceGetStreamPriorityRange'; did you mean 'hipDeviceGetStreamPriorityRange'?
src/core/dflash2.cpp:206:9: error: use of undeclared identifier 'cudaStreamCreateWithPriority'; did you mean 'hipStreamCreateWithPriority'?
src/core/dflash2.cpp:207:9: error: use of undeclared identifier 'cudaStreamCreateWithPriority'; did you mean 'hipStreamCreateWithPriority'?
ninja: build stopped: subcommand failed.
```
The gfx906 build is a backend of its own: its compile lines carry `-DSTRATA_HIP_GFX906=1 -D__HIP_PLATFORM_AMD__=1` and
never `-DSTRATA_USE_HIP` (0 occurrences in the whole build log), so both `#if defined(STRATA_USE_HIP)` branches in this
file were compiled as CUDA. Extending both guards to `#if defined(STRATA_USE_HIP) || defined(STRATA_HIP_GFX906)` makes
the same command link (`[85/85] Linking HIP executable strata`), and the engine has been serving
Qwen3.8-Flash-Next IQ3_XXS on 2x Radeon VII at `--max-context 131072` since.
## Related discussion
Follow-up to **#3986** (open, MarcoTool, branch `gfx906-dflash2-build`): its one hunk is identical to the first hunk
here, and its build check says it was never run on a gfx906 card. **#4156** (KevinX8, closed unmerged) fixes the same
failure a different way, mapping the two calls in `include/strata/platform/hip_compat/strata_hip.h`; the build-check
comment on #3986 compares the two and notes they differ at runtime (call-site guard keeps the default priority unless
`STRATA_DF2_PRIO=1`, the compat shim gives the highest priority by default).
Neither submission touches the second guard in this file — `capture()`'s `fork_default` — which is what is new here,
along with a run on actual gfx906 hardware. If you would rather take #3986, say so and I will drop the first hunk and
rebase the second onto it.
## Approach
Both guards take the form the rest of the tree already uses for this backend: `mtp.cpp:475`, `verify.cpp:219`,
`expert_cache.cpp:6`, `prefill.cpp`, `generate.cpp`, `kv_hybrid_parity.cpp` and several headers all test
`STRATA_USE_HIP` and `STRATA_HIP_GFX906` together.
- `load()` (line 198): gfx906 keeps the HIP behaviour, i.e. the drafter stream takes the default priority unless
`STRATA_DF2_PRIO=1` — the same choice `mtp.cpp` already makes for gfx906
(`#if !defined(STRATA_USE_HIP) && !defined(STRATA_HIP_GFX906) && !defined(STRATA_GPU_METAL)`, "(HIP, Metal: the
default priority)").
- `capture()` (line 446): gfx906 keeps one drafter stream. Without the guard it inherits `fork_default = true`, the
CUDA default that the comment two lines above rules out for HIP ("measured: -0.3 ms a round on an RTX 3060,
+0.0..0.7 on a Strix Halo (gfx1151), so the HIP builds keep one stream"). Vega is nearer that measurement than an
RTX 3060 is. `STRATA_DF2_FORK=1` still opts back in per run.
Compatibility: the change only alters a build that defines `STRATA_HIP_GFX906` and not `STRATA_USE_HIP`, which is the
gfx906 cmake branch (`CMakeLists.txt:199,205`) alone. CUDA builds define neither macro and stay on `#else`; the wave32
HIP build defines `STRATA_USE_HIP=1` (`cmake/hip_backend.cmake:98`) and stays on the same branch it was on. No API,
flag or file changes, no unrelated cleanup.
## Validation
Environment: the ROCm build container (`/opt/rocm/core-7.14`, AMD clang 23.0.0git from `/opt/rocm/llvm`, cmake 3.28.3),
`STRATA_GGML_DIR` = llama.cpp `3cf03257f219`, repo at main `870aedd`. Host: 2x Radeon VII 16 GB (x8 Gen3 each), 90 GiB
RAM, Unraid 7.3.2, container `--ipc=host --shm-size 2g`, `HIP_VISIBLE_DEVICES=0,1`.
**Passed:**
1. Full build after the change —
`cmake -S /work/src/Strata -B /work/build-906 -G Ninja -DSTRATA_HIP_GFX906=ON -DCMAKE_HIP_ARCHITECTURES=gfx906 -DSTRATA_GGML_DIR=/work/src/llama.cpp -DCMAKE_C_COMPILER=/opt/rocm/llvm/bin/clang -DCMAKE_CXX_COMPILER=/opt/rocm/llvm/bin/clang++`
then `ninja -C /work/build-906 strata strata-device`: `[85/85] Linking HIP executable strata`, 0 errors,
`strata` 38,321,920 B, `strata-device` built too. Log `/work/logs/build2.log`.
2. Same translation unit, same flags, unpatched copy vs patched copy (`-fsyntax-only`, tree untouched): **3 errors**
against `git show HEAD:src/core/dflash2.cpp`, **0 errors** against the patched file. This is the negative control, so
the failure is pinned to these two guards and not to something else in the build.
3. The macro claim, read off the build log: `grep -c DSTRATA_USE_HIP /work/logs/build.log` → **0**, while its
dflash2 compile line is
`clang++ -DSTRATA_HAVE_DFLASH2=1 -DSTRATA_HIP_GFX906=1 … -D__HIP_PLATFORM_AMD__=1 -isystem …/include/strata/platform/hip_compat … src/core/dflash2.cpp`.
4. Runtime on gfx906: `strata-device` lists `device 0: AMD Radeon VII`, `device 1: AMD Radeon VII` (both "compute
capability 9.0, 16.0 GiB") with `HSA_OVERRIDE_GFX_VERSION` unset. Serving with
`--pack data/packs/iq3_xxs --native <shard1> --ple-gguf <shard2> --prefill auto --spec 4 --spec-min-p 0.5 --mtp data/mtp/rt --max-context 131072 --kv int8 --kv-resident 32768 --pipeline-windows 2 --layer-split auto`:
~3 h of continuous agent use (engine started 17:06, log still being written at 20:07 local), 55–110 tok/s decode, 100–230 tok/s prefill, expert cache 93–98 % hit,
12–13k experts resident in VRAM (~21 GB), ~15.5 GB VRAM per card, 54–62 °C, ~355 W per card against the 500 W limit,
PCIe Gen3 x8 per card. Logs `/work/logs/strata-iq3_xxs.log`, `/work/logs/device.log`.
**Failed:**
- The same build without the change (`/work/logs/build.log`, errors at log lines 2280/2290/2297): the three errors
quoted above, `ninja: build stopped: subcommand failed`. Nothing else failed — this is the pre-change state, kept as
the control. No failure observed with the change applied.
**Not run:**
- The dflash2 drafter itself. This serving run uses the MTP draft layer (`--mtp …/data/mtp/rt`), not
`--draft dflash2`, and the engine log contains no `dflash` line: the code both guards select is compiled and linked,
but its runtime effect (stream priority, forked context update) was never exercised on a gfx906 card. Nothing in
`docs/AMD_HIP.md` or `docs/OLDER_GPUS.md` documents dflash2 on gfx906 either. So the `capture()` half of this change
rests on the measurement recorded in that comment plus the `mtp.cpp` precedent, not on a new benchmark. I can run
`--draft dflash2` with `STRATA_DF2_FORK=1`/`0` on this pair if the merge decision needs that number.
- The test suite: CMake gates tests out of the gfx906 build (`CMakeLists.txt:691`,
`STRATA_BUILD_TESTS AND STRATA_ENABLE_CUDA AND NOT STRATA_HIP_GFX906`), so no unit or bench targets were built or
run here.
- CUDA, wave32 HIP (`-DSTRATA_ENABLE_HIP=ON`), SYCL and Metal builds: not run, no NVIDIA card and no CUDA toolkit in
this environment. Those paths are untouched by the guard change but not re-verified.
- Other quants/models, single-card gfx906, and Windows: not run.
## Limits
- The `capture()` default is a consistency choice, not a measurement: nobody has measured fork vs no-fork for the
drafter on gfx906, and I did not benchmark it here. `STRATA_DF2_FORK=1` restores the CUDA behaviour per run, so the change stays reversible.
- If the compat-header route (#4156) is preferred for the stream-priority calls, the first hunk here should be dropped
and only the `capture()` guard kept.
- Compiling and linking the drafter on gfx906 is not the same as the drafter working end to end on gfx906 (it also
needs `--draft-model`); this branch says nothing about that.
- Other gfx906 gaps in the repo (#4151 `__shared__ alignas(16) int4`, #2883 reproducible releases) are untouched here.
関連リンク
インストール・モデル・リリースへのサイト内リンク。