#6414 [Engine] gfx906: reference build for MI50/MI60/Radeon VII (0.2.11 + int6 kernels)

open · draft · @romeo5353535353-ui · 0 comments · View on GitHub

BenchmarksMulti-GPUAMD / HIPNVIDIA / CUDAModels & quantsDocumentation

Description

## Problem and result

gfx906 cards (Instinct MI50/MI60, Radeon VII) have no ready-made engine, and on this hardware the
0.2.11 engine decodes at ~60/70 tok/s and reads a 10.5K-token prompt at ~607 tok/s. This branch adds
gfx906 kernels on top of 0.2.11 and measures the result on 2× MI50 16 GB: decode 72–82 tok/s,
prefill 628–1022 tok/s, a 117K needle in 129–136 s (stock 168 s), and a config-only quality mode that
brings the outputs back to the stock reference (KL 0.0009 against 0.0126).

This is a **reference implementation**, not a request to merge 24 commits into main as is.

## Related discussion

- #3172 and #3173 — the 2× MI50 production assistant these numbers come from.
- #4766 — DFlash2 on gfx906; the stream-priority measurements are posted there.
- Scope agreement: none yet. I am happy to split this into focused PRs (the build recipe/docs, or one
  kernel at a time) if a maintainer picks the scope.

## Approach

24 commits on top of v0.2.11: int6 native IQ pack (IQ3_S experts, IQ4_XS dense), repacked expert
GEMMs (MMQ, dense MMQ, IQ GEMV), MMVQ Q4_K/Q6_K, the GR (hyper-connection) port, MTP paths, and a
config-only quality mode (`STRATA_PF_RP=0 STRATA_DENSE_MMQ=0 STRATA_MMQ_KQ=0 STRATA_DENSE_RP=0
STRATA_HIP_PROMPT_F16=0 STRATA_OLD_IQ_MMVQ=1`). Branch `gfx906`; the binary is in the
[v0.2.11-gfx906 release](https://github.com/romeo5353535353-ui/Strata/releases/tag/v0.2.11-gfx906).

## Validation

Environment: revision `5c16b4c0`, 2× Instinct MI50 16 GB, Ryzen 7 5700X, 125 GB RAM,
`mixa3607/rocm-gfx906:7.14-complete`, engine 0.1.57, Qwen3.8-Flash-Next IQ3_S pack,
`--layer-split 24 --spec 2 --prefill 4096 --kv int8`.

**Passed:**
- `cmake -S . -B build-906 -G Ninja -DSTRATA_HIP_GFX906=ON -DCMAKE_HIP_ARCHITECTURES=gfx906 ... && ninja -C build-906 strata` → build OK, `sha256 fe420115…`
- decode 160 greedy tokens, 3 runs: 72–75 tok/s (p0), 82–84 (p2); quality mode 71.7 / 82.3
- prefill 10.5K / 43K: 755 / 1022 tok/s (fast), 628 / 827 (quality)
- needle 117K: 129 s fast, 136 s quality, answer found (stock 168 s)
- KL against the stock 0.2.11 engine over 891 positions: 0.0126 fast, 0.0009 quality (top-1 99.66 %)
- chat and function calling: `get_order_status {"order_id":"4471"}`, finish `tool_calls`

**Failed:**
- none observed on this branch.

**Not run:**
- CUDA, SYCL and Metal builds (no hardware)
- gfx900 / gfx1012 cards (shared path, untested here)
- Qwen3.6 family (not part of this build)
- DFlash2 on this branch (we use MTP; DFlash2 measured slower here)

## Limits

- The branch is 0.2.11-based; 0.2.17 is current. A rebase is not included: 0.2.17 measured slower on
  this box (decode 63/70 tok/s, prefill 593/551) and would need the kernels re-validated.
- The native IQ pack must be built with this branch's packer.
- Fast mode is KL 0.0126 against the stock reference; quality mode trades ~17–19 % prefill for KL 0.0009.
- 24 commits is a fork, not a minimal patch — happy to split it.

Related on this site

Quick links to related pages.