#6404 [Engine] gfx1030 (RX 6800 XT, Windows): 0.1.63 reads prompts 2x faster than 0.1.60 but decodes 11 % slower — same config, A/B on one host
open · @mircolike · 3 comments · View on GitHub
BenchmarksSetup & installServer & APIMulti-GPUAMD / HIPModels & quantsWindows
Description
**Update 2026-11-21 — the numbers in this report were measured on the previous Coder shard1.** Re-run on the shard now installed (`sha256 bfa3644d…`, the #3445 build): decode median **37.0 tok/s** over 6 runs across two sessions (range 35.6–38.3), prefill **1 027 tok/s**, i.e. the same picture as below. The model swap is not what the decode difference comes from; the 0.1.63-vs-0.1.60 gap and the 2× prefill gain both reproduce on the current shard. Details, per-run numbers and a note on draft acceptance in the comment. ### Area [Engine] ### Desktop app, Terminal or source? Desktop app (Strata 0.2.15 GUI) launching the bundled engine. Not built from source. Same host, same model, same settings before and after — only the engine component changed. ### What happened I updated from engine **0.1.60** to **0.1.63** (hip-rdna2, `commit 00157e14b534` → `01d81206d420`) on a **gfx1030** card and measured both sides with the same script. Prompt processing got about **2× faster**, but token generation got about **11 % slower**, consistently and reproducibly. I am filing this because you cannot measure gfx1030 Windows yourselves, and because 0.1.63 added `gfx1032` to the build's arch list — if a kernel choice for the new arch changed the gfx1030 path, you would not have seen it elsewhere. ### Numbers Both engines, identical run config (32/32 Engine-tab values identical between the two runs): `--max-context 65536 --kv k8v4 --kv-resident 32768 --spec 4 --spec-min-p 0.5 --mtp --expert-cache auto --prefill auto --pool-workers 7 --vram-reserve-mib 1536`, context 128K in the Engine tab, KV INT8, draft On depth 4, batch 2048, threads 6, flash attention on. Method: `/v1/chat/completions`, `temperature 0`, numbers read from the response's `timings` (`predicted_per_second` / `prompt_per_second`), not from wall clock. | | 0.1.60 | 0.1.63 | Δ | |---|---|---|---| | decode, short prompt + 600 generated tokens | **42.0 tok/s** (43.3 / 42.0 / 40.1) | **37.5 tok/s** (8 runs: 36.3–40.2, steady 37.3–38.1) | **−11 %** | | prefill, 9 635-token prompt | **504.8 tok/s** (507.0 / 502.5) | **1 067.4 tok/s** (1 069.8 / 1 065.0) | **+111 %** | | drafts accepted (short prompt) | 69–77 % | 72–78 % | unchanged | The decode samples do not overlap: the old engine's slowest run (40.1) is faster than the new engine's fastest (40.2 on one noisy run; 38.1 and below after warm-up). Draft acceptance is unchanged, so this is not the speculative path. ### 0.1.63 at longer contexts (200 generated tokens each) | prompt tokens | prefill | decode | drafts accepted | |---|---|---|---| | ~80 | — | 37.5 | 0.74 | | 2 663 | 857.6 / 847.9 | 32.5 / 34.1 | 0.70 / 0.81 | | 13 674 | 892.8 / 883.9 | 33.8 / 33.1 | 0.74 / 0.71 | | 43 896 | 1 074.2 | 29.2 | 0.65 | | 68 424 | HTTP 400 `request exceeds the context window … leaves no room to answer` | | | Decode falls ~11 % from short prompts to ~2.7k and another ~11 % by 44k — a flat decline, no cliff. Draft acceptance drops to 0.65 at 44k, which explains part of the second step. I did not keep 0.1.60 installed long enough to measure the same length series, so I cannot say whether the long-context shape changed; the short-prompt regression is the solid finding. ### Relation to the other gfx1030 reports This is not the #6264 / #6269 prefill gap: those track the Gyro trellis experts, which fall back to FP16 because the fused prompt kernel needs RDNA3/RDNA4 matrix cores. My pack is a **Coder IQ1_M native pack on Windows**, and on the same prompt length the prefill number went **up, 504.8 → 1 067.4 tok/s**. So whatever fixed prompt reading here works; the cost seems to have moved into the decode path. ### What I would like to know 1. Is the decode difference expected for 0.1.63 on gfx1030 (a deliberate change, e.g. the gfx1032 kernels or a different attention path), or a regression worth bisecting? 2. If it is a regression, is there a flag that puts 0.1.63 back on the 0.1.60 decode path so I can confirm it is the kernel and not the scheduler? I keep the old component directory, so any 0.1.6x build can be A/B'd in about 15 minutes on this machine. ### Smallest steps or command that reproduce it 1. Install engine 0.1.60 (keep the component directory), load the Coder IQ1_M pack on a gfx1030 card, and POST a fixed greedy request with `max_tokens` 600; read `timings.predicted_per_second`. 2. Update to 0.1.63 through the app, load the same pack, POST the same request; read `timings.predicted_per_second` again. 3. Repeat 3 times per build. 0.1.60 gave 43.3 / 42.0 / 40.1; 0.1.63 gave 36.3 / 37.2 / 40.2 in the first pass, and 37.3 / 38.1 / 38.0 / 37.5 / 37.3 in a second pass of 5. Prefill on a 9 635-token prompt moved the other way, 504.8 → 1 067.4 tok/s. ### Reproduction status Consistently reproducible — 6 decode runs on 0.1.63 (steady 37.3–38.1 after warm-up) against 3 on 0.1.60 (40.1–43.3); the two ranges do not overlap. The 32 Engine-tab values were identical between the two measurements, the model file and the launch args were identical; only the engine component changed. ### Strata version Strata 0.2.15 · engine 0.1.60 → 0.1.63 (`windows-x86_64-hip-rdna2`, commit `00157e14b534` → `01d81206d420`) · desktop 0.2.15 ### Graphics card, memory and system RX 6800 XT 16 GB (**gfx1030**) · 47.4 GB RAM · Windows 11 · pagefile 7 GB (commit limit ≈ 54.6 GiB) · driver 32.0.21045.29003 ### Model, size and settings Qwen3.8-Flash-Next **Coder IQ1_M** (native pack, 2 files + separate MTP) · `--max-context 65536` · `--kv k8v4` · `--spec 4` · engine 0.1.63, desktop 0.2.15
Related on this site
Quick links to related pages.