#6408 [Docs] Community benchmark: one Radeon VII — ROCm 10.1 vs 7.14, engine 0.1.63 vs 0.1.62
open · @Zuzik01 · 0 comentarios · En GitHub
BenchmarksAMD / HIPNVIDIA / CUDAModels & quantsDocumentationWindows
Descripción
## Problem and result `docs/bench-results/` has reports from two-MI50 boxes but none from a single Radeon VII, and no run that separates the ROCm version from the engine version on gfx906. The gfx906 build is community-tested and the maintainers have no such card, so this adds one card's measurements in the usual format. Before: no single-card gfx906 numbers, no ROCm 10.1 against 7.14 on one card, and no comparison of engine 0.1.63 against 0.1.62 on gfx906. After: five `standard-v1` runs on one Radeon VII (16 GiB, gfx906, wave64), the same model (Qwen3.8-Flash-Next IQ3_XXS), the same configuration (context 32768, KV int8, expert cache auto, prefill 2048) and the same protocol; each folder carries its README, `results.json`, `outputs.jsonl` and provenance. All five runs completed: 21 of 21 requests, no failures, 0 skipped. Runs: ROCm 7.14 and ROCm 10.1 on engine 0.1.62, and engine 0.1.63 (v0.2.17) on 7.14 — each with and without the PLE table (26.8 GiB) locked in RAM. ## Related discussion #6246 — the two stream-priority names; now in `main` (`65207788`), so v0.2.17 builds gfx906 with no local patch. #4671 — the PLE table lock, which is where most of the difference below comes from. #4154 — the two-MI50 ROCm report; this is the single-card case. ## Approach Reports only, no code. One folder with five subfolders plus a line in `docs/bench-results/README.md`. ## Validation Each run: `strata bench run --url http://127.0.0.1:8095 --suite standard-v1`, reports as written by the tool, medians of three repetitions per case. **Passed** — all five runs: 21/21 requests, 0 failed, 0 skipped. Key numbers (decode tok/s, medians): | Case | 7.14 / 0.1.62 | 7.14 / 0.1.62 locked | 10.1 / 0.1.62 | 7.14 / 0.1.63 | 7.14 / 0.1.63 locked | | --- | ---: | ---: | ---: | ---: | ---: | | short | 30.3 | 31.2 | 31.7 | 27.9 | 28.6 | | prompt-4k | 26.4 | **29.9** | 28.0 | 23.5 | 26.0 | | prompt-16k | 28.2 | **30.6** | 27.0 | 25.1 | 26.1 | | sanity | 12/12 | 12/12 | 12/12 | 11/12 (R1) | 11/12 (I1) | 1. ROCm 10.1 (`10.1-complete`, clang 24) is not faster than 7.14 (`7.14-complete`, clang 23) on one gfx906 card: prompt reading within ±1 % at 16K (about 7 % lower at 4K, one reproduction at 240.9 aside), decode within the spread of the repetitions. 10.1 is a way to stay on a maintained base, not a speedup. 2. Locking the PLE table helps a 16 GiB card: 4K decode 26.4 → 29.9 (+13 %), 16K decode 28.2 → 30.6 (+9 %) on 0.1.62, prompt speed and TTFT unchanged, sanity still 12/12. The table is larger than the card's memory, so its pages are exactly the ones that get reclaimed when it is not locked. 3. Engine 0.1.63 is slower here: decode 8-11 % lower without the lock and 6-7 % lower with it, prompt unchanged, one sanity check failing per run — R1 (answer hit the output cap) unlocked, I1 (a JSON value differed) locked. 0.1.62 passed 12/12 twice on the same day. The 0.1.63 log shows the expert arena read through the file cache. **Failed** — no run failed; the two 0.1.63 sanity failures above are recorded in their own reports. **Not run** — CUDA, SYCL, Metal, Windows, and any multi-card configuration; no other model or quantization. ## Limits One card, one host, one model, three repetitions per case. Cache and page state were not controlled beyond the fixed configuration, so differences of a few percent are indicative rather than exact. Building on `10.1-complete` needs an extra layer: that image ships without cmake, ninja and the python packages the terminal needs (`yaml`, `jinja2`, `regex`, `numpy`, `PIL`, `requests`, `psutil`).
En el sitio
Enlaces a install, modelos, releases.