#6622 [Engine] gfx906 platinum graft: rp prompt-path stack on v0.2.14.1 — 148 tok/s decode on 4x MI50
open · @rpcribari · 0 comments · View on GitHub
BenchmarksServer & APIAMD / HIPNVIDIA / CUDAModels & quantsDocumentation
Description
## What this is
The **rp-graft** prompt-path stack for gfx906, now running our daily-driver production build ("platinum") on 4× MI50 since 2026-11-15:
| probe | result |
|---|---|
| decode_128 / decode_256 (bench) | **148–149 tok/s** |
| prefill_8192 (bench) | **1290–1305 tok/s** |
| long cold prefill (60–70K-token prompts, live traffic) | **~2150–2190 tok/s** |
| generation, real-world average (Strata monitor, 950K tokens generated) | **68 tok/s** at 50–115K-token contexts |
Previous community reference points on this hardware class: stock upstream engines regress gfx906 decode ~40 % from the 0.1.60 generation onward; the int6 line (#6414) measures 72–82 tok/s on 2× MI50. This graft roughly doubles that on 4× MI50 at 262K context, HBM-resident.
## Contents
Branch `rp-graft-0214` = tag `v0.2.14.1` + 13 graft commits (+3851 lines, 20 files) + 2 publishing commits. The stack (each part env-switchable, `=0` turns it off):
- `STRATA_PF_RP` — IQ4_NL expert products through mx-llama.cpp's repacked GEMM (vendored under MIT in `src/prefill/rp/`)
- `STRATA_DENSE_MMQ` / `STRATA_DENSE_RP` — dense projections through int8 MMQ / repacked GEMM (IQ4_NL, IQ4_XS, Q8_0, Q6_K, Q5_K; bitwise-equal checks against stock MMQ included)
- `STRATA_MMQ_KQ` + expert repacks per product (IQ3_XXS / IQ3_S gate/up)
- `STRATA_HIP_PROMPT_F16`, gfx906 build patches, `dense_mmq_bench` / `rp_expert_bench` verification tools
Built with the clang 24 bundled in ROCm 10.1 on `mixa3607/rocm-gfx906:10.1-complete` (the +8 % over clang-23 builds is compiler-only — same source).
## Why the base is v0.2.14.1
We measured every upstream release 0.2.15 → 0.2.19 for graftability: the engine-core changes are CUDA/RDNA-targeted and the 0.1.60+ generation carries the gfx906 decode regression, so a rebase today would graft onto a slower core. Happy to rebase onto main once the wave64 path stops regressing (or can point a maintainer at the exact commits where it starts). Expect conflicts against current main for the same reason — the three-dot diff is the clean graft.
**Supersedes #5361** (the int6-era branch, 125 tok/s) — that PR will be updated to point here.
## Reproduce / adopt
Prebuilt engine binary (sha256 `accc4ac0…`), exact build recipe, serve config, bring-up guide and a production-monitor screenshot: **[`rpcribari/Strata` release `platinum-r101`](https://github.com/rpcribari/Strata/releases/tag/platinum-r101)** + [`README.md`](https://github.com/rpcribari/Strata/tree/rp-graft-0214). Hardware envelope (PPTable + 1080 MHz memclock cap, NUMA pins) is documented there — the numbers assume it.
cc @webzone (strata-gfx906) — thought this might be useful for your rebase work.
Related on this site
Quick links to related pages.