#6552 [Engine] gfx1100 (RX 7900 GRE): 0.2.16 cold-prefill gap is driver pauses, not kernels — 4 "Linux moved memory" pauses/25min vs 0 on 0.1.57

open · @lsunay · 4 comentários · No GitHub

Setup & installServer & APIAMD / HIPModels & quantsLinux

Descrição

### What did you measure, or what went wrong

**TL;DR** On a real RX 7900 GRE (gfx1100, Linux), the cold-prefill gap between engine 0.1.57 (local HIP build, the 0.2.11 line) and 0.1.64 (official `linux-x86_64-hip-rx7900` package, 0.2.18) is mostly explained by `the graphics driver paused the GPU ... while Linux moved memory the GPU uses` pauses: **4 pauses / 13.6 s in a 25-minute matched run on 0.1.64, 0 on 0.1.57**. With the pauses excluded, the 16K cold-prefill rate gap is only about **−4 %** (≈894 vs 930 t/s), not −30 %. `STRATA_HIPBLASLT_TABLE=on` (the winning arm of #6429) did **not** help with a UD-IQ3_XXS pack on either engine. Power draw during the regression was *equal* (175 vs 184 W mean), so the efficiency loss (~−26 %) is not "more watts" — it is the same watts doing less work.

**Setup (identical except engine binary + the table env)**
- RX 7900 GRE 16 GB (gfx1100), X570 dual-CCD Ryzen, 94 GiB RAM, Debian 13 / Linux 6.12, `kernel.numa_balancing=0`
- Model: Qwen3.8-Flash-Next Unsloth **UD-IQ3_XXS** (4 shards), 262144 ctx, `--kv int8 --kv-resident 32768`, `--spec 4 --spec-min-p 0.5`, `--mtp rt`, `--expert-profile` + `--expert-cache auto`, API server (terminal/serve/server.py)
- Arm A: engine 0.1.57 local HIP build (source e098a67d, venv ROCm 7.2 SDK → hipBLASLt **100200**)
- Arm B: engine 0.1.64 official rx7900 full package (binary + bundled tuning) — same pack, same quant, same run scripts
- Harness: 3 fixed prompts (111 / 554 / 16350 tokens) × 2 rounds, fresh engine restart before each arm, warm-up round discarded, test-only request filter, `rocm-smi` package-power sampling every 2 s, no other traffic

**Results**

| | 0.1.57 default | 0.1.57 + `TABLE=on` | 0.1.64 + `TABLE=on` |
|---|---|---|---|
| 16350-tok cold prefill | 930.3 t/s | 933.5 t/s | **654.4 t/s** (8.8 s pause inside; ≈894 excl. pause) |
| decode run-total | 41.1 t/s | 40.6 t/s | 38.8 t/s |
| driver pauses | **0** | **0** | **4 (1196/2380/8772/1244 ms)** |
| draft acceptance (big arm, same `--mtp rt`) | 57.6 % | — | 51.1 % |
| expert-cache hit | 78.2 % | — | 74.3 % |
| package power mean/max | 184/225 W | 178/222 W | 175/224 W |

The table is confirmed loaded on 0.1.64: `prefill gemm: hipBLASLt tuning enabled (26 rows, gfx1100, version 100200)`.

**Questions**
1. Is the `Linux moved memory the GPU uses` pause expected under the new memory scheme (0.2.17: 24→17 GiB reserve, expert 47 GiB)? Is there an engine knob (populate / pin-all / madvise variant) to keep Linux from migrating the GPU's pages on a 16 GB gfx1100 host?
2. The #6429 table benefit (1.33–1.89× at ≥4K) does not reproduce on **UD-IQ3_XXS** (930.3 → 933.5, noise, same hipBLASLt 100200 as the winning arm). Are the 26 rows specific to GSQ-RCO/IQ3_S dense GEMM shapes (cf. #5581)?
3. Draft acceptance fell 57.6 % → 51.1 % with the same MTP pack on 0.1.64 — did the verify/draft path change for gfx1100?

### Reproduction status

Measured locally on one RX 7900 GRE: matched-pair harness (3 prompts × 2 rounds, fresh restart per arm, same day), plus the 0.1.64 pause behaviour reproduced in an earlier overnight session (2 pauses there). Raw logs, `watt.csv` and per-request JSON available on request.

No site

Links install, modelos, releases.