Issues / #432

#432 Benchmark report: R9700 (gfx1201) on 0.1.32 with TheRock 7.10 — router +12% not reproduced, HIP_WMMA prompt +29-34%

closed · @jkuepker · 3 comentarios · En GitHub

BenchmarksSetup & installServer & APIAMD / HIPModels & quantsSecurityLinux

Descripción

This is a report of numbers measured against the release-note claims, not a bug report. All runs used the same Coder IQ1_M pack on one R9700. localeval streamed sweeps: fresh nonce each call (no prefix cache), 128 forced output tokens, temperature 0, median of reps after a warmup. The server was restarted for every configuration.

| Config | Prompt 4K (tok/s) | Prompt 16K (tok/s) | Decode 128 / 1K / 4K (tok/s) |
|---|---:|---:|---|
| default | 1,331 | 1,310 | 91.0 / 75.3 / 68.0 |
| `STRATA_HIP_ROUTER_OLD=1` | 1,215 | 1,312 | 81.7 / 69.6 / 68.7 |
| `STRATA_HIP_WMMA=1` | **1,712 (+29%)** | **1,759 (+34%)** | 85.2 / 70.0 / 68.8 |
| `STRATA_SELECT_WMMA=1` | 1,223 | 1,324 (+1%) | 88.7 / 70.2 / 65.3 |
| `--pool-affinity auto` (12600K P/E cores) | 1,336 | 1,314 | 82.9 / 74.3 / 64.2 |

**Router A/B/A/B rerun (6 reps each, decode tok/s):**

| Run | 128 | 1K | 4K |
|---|---:|---:|---:|
| new | 91.9 | 71.2 | 63.3 |
| old | 90.7 | 70.5 | 66.3 |
| new | 90.6 | 74.5 | 68.1 |
| old | 89.4 | 69.7 | 62.8 |

- **Router:** the new router averages +1-4% decode. The spread within a run is 12-20%, so I can't detect the +12% (62 → 70 tok/s) the notes report. I confirmed the engine received `STRATA_HIP_ROUTER_OLD=1` by reading `/proc/<pid>/environ`. Greedy output is bit-identical between the two routers across 10 server starts, which matches the notes.
- **HIP_WMMA:** a clear gain, but below the +36-51% in the notes. The greedy 256-token output hash did not change on a short prompt.
- **SELECT_WMMA:** +1% at 16K, in line with the +1.5% in the notes.
- **A difference that may matter:** the release numbers came from ROCm 10.2 nightly with hipBLASLt 1.5.0. This box uses what setup installs, TheRock 7.10 wheels with hipBLASLt 1.2.0, so the shipped gfx1201 tuning table doesn't load. Another difference is the CPU: an i5-12600K here, versus 16 threads on the author's machine.

Question: what prompt length and which settings produced the 62 → 70 tok/s figure? With those I can rerun under the same conditions.

**Environment**

```
Strata v0.1.32 (tag c499bd1), Linux, Ubuntu 24.04.4, kernel 7.0.0-34
GPU: Radeon AI PRO R9700 32 GB (gfx1201), single card
CPU: i5-12600K (AVX2, no AVX-512), 62 GB DDR5
Setup: ./setup.sh --backend hip --family coder --model IQ1_M (no system ROCm)
ROCm: TheRock 7.10.0a20251120 wheels (gfx120X-all) installed by setup into .venv; hipBLASLt 1.2.0
Server: serve/server.py with setup's generated config (max-context 131072, kv int8, spec 4 + MTP)
```

En el sitio

Enlaces a install, modelos, releases.