#6408 [Docs] Community benchmark: one Radeon VII — ROCm 10.1 vs 7.14, engine 0.1.63 vs 0.1.62

open · @Zuzik01 · 0 comentarios · En GitHub

BenchmarksAMD / HIPNVIDIA / CUDAModels & quantsDocumentationWindows

Descripción

## Problem and result

`docs/bench-results/` has reports from two-MI50 boxes but none from a single Radeon VII, and no run that separates
the ROCm version from the engine version on gfx906. The gfx906 build is community-tested and the maintainers have
no such card, so this adds one card's measurements in the usual format.

Before: no single-card gfx906 numbers, no ROCm 10.1 against 7.14 on one card, and no comparison of engine 0.1.63
against 0.1.62 on gfx906.
After: five `standard-v1` runs on one Radeon VII (16 GiB, gfx906, wave64), the same model (Qwen3.8-Flash-Next
IQ3_XXS), the same configuration (context 32768, KV int8, expert cache auto, prefill 2048) and the same protocol;
each folder carries its README, `results.json`, `outputs.jsonl` and provenance. All five runs completed: 21 of 21
requests, no failures, 0 skipped.

Runs: ROCm 7.14 and ROCm 10.1 on engine 0.1.62, and engine 0.1.63 (v0.2.17) on 7.14 — each with and without the
PLE table (26.8 GiB) locked in RAM.

## Related discussion

#6246 — the two stream-priority names; now in `main` (`65207788`), so v0.2.17 builds gfx906 with no local patch.
#4671 — the PLE table lock, which is where most of the difference below comes from.
#4154 — the two-MI50 ROCm report; this is the single-card case.

## Approach

Reports only, no code. One folder with five subfolders plus a line in `docs/bench-results/README.md`.

## Validation

Each run: `strata bench run --url http://127.0.0.1:8095 --suite standard-v1`, reports as written by the tool,
medians of three repetitions per case.

**Passed** — all five runs: 21/21 requests, 0 failed, 0 skipped.

Key numbers (decode tok/s, medians):

| Case | 7.14 / 0.1.62 | 7.14 / 0.1.62 locked | 10.1 / 0.1.62 | 7.14 / 0.1.63 | 7.14 / 0.1.63 locked |
| --- | ---: | ---: | ---: | ---: | ---: |
| short | 30.3 | 31.2 | 31.7 | 27.9 | 28.6 |
| prompt-4k | 26.4 | **29.9** | 28.0 | 23.5 | 26.0 |
| prompt-16k | 28.2 | **30.6** | 27.0 | 25.1 | 26.1 |
| sanity | 12/12 | 12/12 | 12/12 | 11/12 (R1) | 11/12 (I1) |

1. ROCm 10.1 (`10.1-complete`, clang 24) is not faster than 7.14 (`7.14-complete`, clang 23) on one gfx906 card:
   prompt reading within ±1 % at 16K (about 7 % lower at 4K, one reproduction at 240.9 aside), decode within the
   spread of the repetitions. 10.1 is a way to stay on a maintained base, not a speedup.
2. Locking the PLE table helps a 16 GiB card: 4K decode 26.4 → 29.9 (+13 %), 16K decode 28.2 → 30.6 (+9 %) on
   0.1.62, prompt speed and TTFT unchanged, sanity still 12/12. The table is larger than the card's memory, so its
   pages are exactly the ones that get reclaimed when it is not locked.
3. Engine 0.1.63 is slower here: decode 8-11 % lower without the lock and 6-7 % lower with it, prompt unchanged,
   one sanity check failing per run — R1 (answer hit the output cap) unlocked, I1 (a JSON value differed) locked.
   0.1.62 passed 12/12 twice on the same day. The 0.1.63 log shows the expert arena read through the file cache.

**Failed** — no run failed; the two 0.1.63 sanity failures above are recorded in their own reports.

**Not run** — CUDA, SYCL, Metal, Windows, and any multi-card configuration; no other model or quantization.

## Limits

One card, one host, one model, three repetitions per case. Cache and page state were not controlled beyond the
fixed configuration, so differences of a few percent are indicative rather than exact. Building on
`10.1-complete` needs an extra layer: that image ships without cmake, ninja and the python packages the terminal
needs (`yaml`, `jinja2`, `regex`, `numpy`, `PIL`, `requests`, `psutil`).

En el sitio

Enlaces a install, modelos, releases.