#6435 [Benchmark] RX 9070 (gfx1201, Windows 11, 32 GB RAM) / Gyro-S TQ1_0 / 0.2.6 → 0.2.17: 0.2.17 defaults lose ~30% prompt and ~12% decode on a 32 GB PC; two settings bring it back

open · @grykom · 4 comments · View on GitHub

BenchmarksSetup & installAMD / HIPModels & quantsWindowsLinux

Description

## Summary

One RX 9070 16 GB on Windows, the same model and preset since 0.2.6, measured after every update we installed. Between 0.2.6 and 0.2.16 the cold 55K prompt went from 726 to ~1524 tok/s and decode from 46 to ~71 tok/s.

On 0.2.17 (engine 0.1.63) with the new defaults this PC gets ~1057 tok/s prompt (731–1445 across 5 fresh starts) and ~62 tok/s decode. Two causes, both visible in the engine log:

1. **RAM headroom.** The engine now keeps 4 GiB free (`room for the complement: 15.40 GiB (19.65 GiB available RAM minus 4 GiB headroom and a 0.25 GiB margin)`). The resident expert complement drops from 18.1–18.4 GiB (0.1.62) to 15.4–17.0 GiB, and only 0–1144 of 2200 lendable slots stay in RAM (`FileExpertSource: WARNING: … will read their experts from the pack during a long prompt`). `STRATA_RESIDENT_HEADROOM_GIB=1` gives 18.3 GiB and 2200/2200 again, the same as 0.1.62.
2. **Route tail skip is off by default** (it was on for RX 9000 in 0.2.16). Decode drops from ~71 to ~62 tok/s. `STRATA_ROUTE_TAIL_SKIP=1` restores it, and on 0.1.63 it is faster than before.

With both settings: prompt ~1422 tok/s (−7% vs 0.2.16), decode ~84 tok/s (+19% vs 0.2.16), 14/14 fresh starts without a stall, and the standard benchmark is `complete` with 12/12 sanity checks. This may affect other 32 GB PCs where the model only just fits in RAM.

## Setup

- GPU: Radeon RX 9070 (non-XT) 16 GB, gfx1201. CPU: Ryzen 7 7800X3D. RAM: 32 GB. Windows 11 Pro, official `windows-x86_64-hip-rx9070` engine packages, updated with `strata update`.
- Model: Qwen3.8-Flash-Next Gyro-S TQ1_0. App preset: 64K context, `--kv int8`, vision off, `--tune skip`, `STRATA_HIP_WMMA=1`, `--drafting dflash2` (MTP until 0.2.9).
- Commit limit warning is present on this PC (page file not changed); no silent exit in any run.

## Method

- Every run from a freshly started engine (`strata launch --preset … --yes --no-tui`), then one cold prompt of 54,669 tokens (64 tokens out) and, in the first start of each variant, 3× a short prompt with 1,024 tokens out. Prompt tok/s and decode tok/s from the server timings. Mean of 3–5 fresh starts.
- Measured over RDP with nothing else on the GPU. With the Windows desktop rendered on the same card and normal apps open, prompt was ~7% and decode ~11% lower (0.2.9, A/B), so all numbers below are RDP.
- Not run: other models, other context sizes, the standard benchmark on 0.2.17 defaults (only on the final settings, below).

## Results (cold 55K prompt / decode 1,024 tokens, tok/s)

| Date | App / engine | Drafter | Prompt 55K | Decode | Notes |
|---|---|---|---:|---:|---|
| 10-26 | 0.2.6 / 0.1.50 | MTP | 726 | 46.2 | first run |
| 10-28 | 0.2.7 / 0.1.52 | MTP | 967 | 64.5 | |
| 11-03 | 0.2.10 / 0.1.56 | DFlash2 | 1168 | 61.5 | MTP: 980 / 59.5 |
| 11-18 | 0.2.15 / 0.1.60 | DFlash2 | ~1454 | 67.9 | MTP: ~1109 / 62.4 |
| 11-19 | 0.2.16 / 0.1.62 | DFlash2 | ~1524 | 70.7 | MTP: ~1139 / 68.2 |
| 11-20 | 0.2.17 / 0.1.63, defaults | DFlash2 | ~1057 (731–1445) | 62.2 | 5 starts |
| 11-20 | 0.2.17 + `ROUTE_TAIL_SKIP=1` | DFlash2 | ~1249 | 77.8 | 3 starts |
| 11-20 | 0.2.17 + `RESIDENT_HEADROOM_GIB=1` | DFlash2 | ~1298 | 62.2 | 3 starts |
| 11-20 | **0.2.17 + both** | DFlash2 | **~1422** (1388–1439) | **84.2** | 3 starts |

## Standard benchmark (0.2.17 + both settings)

`strata bench run --url http://127.0.0.1:8080 --track custom` on a freshly started engine, protocol `strata-qwen38-thinking-v1-draft.1`, Strata commit `12d491da`, engine 0.1.63, nothing else on the GPU (RDP). **Status: complete**, 21 of 21 required requests, 0 failed, **sanity 12/12** (instruction 4/4, retrieval 4/4, tool calls 4/4), 2 min 46 s after load.

| Case | Prompt tok/s | Decode tok/s | TTFT s |
|---|---:|---:|---:|
| short | n/a | 73.8 (68.1–76.5) | 0.69 |
| prompt-4k | 919.8 (913.2–927.3) | 76.5 (66.1–82.3) | 4.49 |
| prompt-16k | 1294.6 (1266.2–1297.6) | 73.2 (65.1–76.5) | 12.69 |

Median (min–max), n=3. Notes: track custom because of the two settings and `--drafting dflash2`; the runner reports that the server had answered 1 request before the run; quantization not read from the file name (Gyro-S TQ1_0). Happy to open a results-only PR with the folder if useful.

Agentic use: we run five private coding tasks (OpenCode, hidden tests) after each update. All passed on every version from 0.2.7 on, with 0 fix rounds. Total time went from 14.7 min (0.2.6) to 8.5–9.7 min (0.2.16 / 0.2.17 + both), but one run varies more than the versions do. In real agent sessions on 0.2.17 + both we now see decode near 60 tok/s, which we did not see before (observation, not measured).

## Linux on the same PC (old engine)

On 2026-11-07, Ubuntu 24.04 live USB, Strata Terminal 0.1.58 with the `linux-x86_64-hip-rx9070` engine, no desktop, same model and settings: a cold 52K prompt ran at ~2026 tok/s against ~1130 on Windows with the same engine 0.1.58 (~1.8×). `strata bench run` standard-v1: complete, 12/12 sanity; prompt 4K 1529, 16K 2176 tok/s; decode 69–75 tok/s. Against the Windows standard run above (0.1.63 + both settings: 920 / 1295 tok/s, decode 73–77) the old Linux engine still reads prompts ~1.7× faster at the same decode speed. Not measured on newer engines on Linux.

## Questions

- Is 4 GiB headroom intended on a 32 GB PC where the whole complement fitted before? A lower default for small-RAM PCs, or a hint in the app, would help.
- Is route tail skip off by default on RDNA4 on purpose (output differences)? Here it is the larger of the two decode effects.

---
*This report was drafted by an AI assistant (Claude) from measurements run on my machine. I reviewed it before posting.*

Related on this site

Quick links to related pages.