#6404 [Engine] gfx1030 (RX 6800 XT, Windows): 0.1.63 reads prompts 2x faster than 0.1.60 but decodes 11 % slower — same config, A/B on one host

open · @mircolike · 3 Kommentare · Auf GitHub

BenchmarksSetup & installServer & APIMulti-GPUAMD / HIPModels & quantsWindows

Beschreibung

**Update 2026-11-21 — the numbers in this report were measured on the previous Coder shard1.** Re-run on the shard now installed (`sha256 bfa3644d…`, the #3445 build): decode median **37.0 tok/s** over 6 runs across two sessions (range 35.6–38.3), prefill **1 027 tok/s**, i.e. the same picture as below. The model swap is not what the decode difference comes from; the 0.1.63-vs-0.1.60 gap and the 2× prefill gain both reproduce on the current shard. Details, per-run numbers and a note on draft acceptance in the comment.

### Area

[Engine]

### Desktop app, Terminal or source?

Desktop app (Strata 0.2.15 GUI) launching the bundled engine. Not built from source. Same host, same model, same settings before and after — only the engine component changed.

### What happened

I updated from engine **0.1.60** to **0.1.63** (hip-rdna2, `commit 00157e14b534` → `01d81206d420`) on a **gfx1030** card and measured both sides with the same script. Prompt processing got about **2× faster**, but token generation got about **11 % slower**, consistently and reproducibly.

I am filing this because you cannot measure gfx1030 Windows yourselves, and because 0.1.63 added `gfx1032` to the build's arch list — if a kernel choice for the new arch changed the gfx1030 path, you would not have seen it elsewhere.

### Numbers

Both engines, identical run config (32/32 Engine-tab values identical between the two runs):

`--max-context 65536 --kv k8v4 --kv-resident 32768 --spec 4 --spec-min-p 0.5 --mtp --expert-cache auto --prefill auto --pool-workers 7 --vram-reserve-mib 1536`, context 128K in the Engine tab, KV INT8, draft On depth 4, batch 2048, threads 6, flash attention on.

Method: `/v1/chat/completions`, `temperature 0`, numbers read from the response's `timings` (`predicted_per_second` / `prompt_per_second`), not from wall clock.

| | 0.1.60 | 0.1.63 | Δ |
|---|---|---|---|
| decode, short prompt + 600 generated tokens | **42.0 tok/s** (43.3 / 42.0 / 40.1) | **37.5 tok/s** (8 runs: 36.3–40.2, steady 37.3–38.1) | **−11 %** |
| prefill, 9 635-token prompt | **504.8 tok/s** (507.0 / 502.5) | **1 067.4 tok/s** (1 069.8 / 1 065.0) | **+111 %** |
| drafts accepted (short prompt) | 69–77 % | 72–78 % | unchanged |

The decode samples do not overlap: the old engine's slowest run (40.1) is faster than the new engine's fastest (40.2 on one noisy run; 38.1 and below after warm-up). Draft acceptance is unchanged, so this is not the speculative path.

### 0.1.63 at longer contexts (200 generated tokens each)

| prompt tokens | prefill | decode | drafts accepted |
|---|---|---|---|
| ~80 | — | 37.5 | 0.74 |
| 2 663 | 857.6 / 847.9 | 32.5 / 34.1 | 0.70 / 0.81 |
| 13 674 | 892.8 / 883.9 | 33.8 / 33.1 | 0.74 / 0.71 |
| 43 896 | 1 074.2 | 29.2 | 0.65 |
| 68 424 | HTTP 400 `request exceeds the context window … leaves no room to answer` | | |

Decode falls ~11 % from short prompts to ~2.7k and another ~11 % by 44k — a flat decline, no cliff. Draft acceptance drops to 0.65 at 44k, which explains part of the second step. I did not keep 0.1.60 installed long enough to measure the same length series, so I cannot say whether the long-context shape changed; the short-prompt regression is the solid finding.

### Relation to the other gfx1030 reports

This is not the #6264 / #6269 prefill gap: those track the Gyro trellis experts, which fall back to FP16 because the fused prompt kernel needs RDNA3/RDNA4 matrix cores. My pack is a **Coder IQ1_M native pack on Windows**, and on the same prompt length the prefill number went **up, 504.8 → 1 067.4 tok/s**. So whatever fixed prompt reading here works; the cost seems to have moved into the decode path.

### What I would like to know

1. Is the decode difference expected for 0.1.63 on gfx1030 (a deliberate change, e.g. the gfx1032 kernels or a different attention path), or a regression worth bisecting?
2. If it is a regression, is there a flag that puts 0.1.63 back on the 0.1.60 decode path so I can confirm it is the kernel and not the scheduler?

I keep the old component directory, so any 0.1.6x build can be A/B'd in about 15 minutes on this machine.

### Smallest steps or command that reproduce it

1. Install engine 0.1.60 (keep the component directory), load the Coder IQ1_M pack on a gfx1030 card, and POST a fixed greedy request with `max_tokens` 600; read `timings.predicted_per_second`.
2. Update to 0.1.63 through the app, load the same pack, POST the same request; read `timings.predicted_per_second` again.
3. Repeat 3 times per build. 0.1.60 gave 43.3 / 42.0 / 40.1; 0.1.63 gave 36.3 / 37.2 / 40.2 in the first pass, and 37.3 / 38.1 / 38.0 / 37.5 / 37.3 in a second pass of 5. Prefill on a 9 635-token prompt moved the other way, 504.8 → 1 067.4 tok/s.

### Reproduction status

Consistently reproducible — 6 decode runs on 0.1.63 (steady 37.3–38.1 after warm-up) against 3 on 0.1.60 (40.1–43.3); the two ranges do not overlap. The 32 Engine-tab values were identical between the two measurements, the model file and the launch args were identical; only the engine component changed.

### Strata version

Strata 0.2.15 · engine 0.1.60 → 0.1.63 (`windows-x86_64-hip-rdna2`, commit `00157e14b534` → `01d81206d420`) · desktop 0.2.15

### Graphics card, memory and system

RX 6800 XT 16 GB (**gfx1030**) · 47.4 GB RAM · Windows 11 · pagefile 7 GB (commit limit ≈ 54.6 GiB) · driver 32.0.21045.29003

### Model, size and settings

Qwen3.8-Flash-Next **Coder IQ1_M** (native pack, 2 files + separate MTP) · `--max-context 65536` · `--kv k8v4` · `--spec 4` · engine 0.1.63, desktop 0.2.15

Mehr auf der Site

Links zu Install, Modellen, Releases.