#3398 [Benchmark] Windows Strix Halo (gfx1151) suite on 0.1.45: --mmap-experts + STRATA_SH_STREAM=0 decode +12-15%

说明

# [Benchmark] Strix Halo (gfx1151) suite on 0.1.45: --mmap-experts + STRATA_SH_STREAM=0 decode +12-15%, UD-IQ4_XS prefill fixed, shared-GPU-memory placement idea

Measured on 2026-10-19 / 2026-10-20 by `mistral13`.

Follow-up to [#3032](https://github.com/Niko1221/Strata/issues/3032) (my first suite report, engine 0.44.1). Same Strix Halo machine, same test suite
(re-run with the engine's own timings), now on **Strata 0.1.45** with the bug fix from
[#3098](https://github.com/Niko1221/Strata/pull/3098) used locally. Coder IQ1_M was replaced by
**qwen3.6-35b-a3b-ud-q6_k_xl**, and the setup takes the port-handling findings from
[#3029](https://github.com/Niko1221/Strata/pull/3029) into account (the suite pins one port for config, server and
bench call and kills any stale engine before every step, so no run can accidentally measure a leftover server).

Speed only — no correctness or recall check was run. Main new results:

- **`--mmap-experts` combined with `STRATA_SH_STREAM=0` is the fastest decode on this APU for both Flash-Next
  quants**: unsloth-ud-iq4_xs chat decode 34.3 → 38.5 tok/s (+12.2 %), and with `--pool-workers 6` 39.6 tok/s
  (+15.5 %); iq3_xxs 34.7 → 39.4 tok/s (+13.5 %), best agent decode 44.7 tok/s. Note `docs/STRIX_HALO.md` lists
  `STRATA_SH_STREAM=1` as the gfx1151 default (it paid +1.8 % / +6.7 % on 0.1.40) — on 0.1.45 **turning it off**
  measured clearly faster, so the per-PC calibration or the doc may want a look.
- **The big prefill gap of the first suite is gone**: unsloth-ud-iq4_xs prompt was 322–355 tok/s on 0.44.1 (much
  worse than iq3_xxs and coder-iq1_m then) and now measures 643–665 tok/s — as fast as the other models. Prompt
  speed is still strongly model-dependent: qwen3.6-35B falls from 596 tok/s (4K) to 373 tok/s (32K) while both
  Flash-Next quants stay flat at 620–665.
- qwen3.6-35b-a3b-ud-q6_k_xl gives the **best chat decode (50.5 tok/s, TTFT 0.25 s)** of the whole suite, but its
  **agent conversation-cache hit is 0 %** (30k tokens re-read per session, per-turn TTFT grows to ~27 s) while both
  Flash-Next packs hit ~68.7 % — looks specific to this arch/model path, details below.
- Placement idea measured out (bottom of this report): on this unified-memory APU the mmap variants keep **almost
  nothing in "shared GPU memory" (0.9 GiB) while process RAM is nearly full (41.5 GiB)** — see
  [shared-memory placement idea](#possible-improvement-shared-gpu-memory-instead-of-normal-ram).

## Hardware and software

- GPU and VRAM: AMD Radeon(TM) 8060S Graphics (Strix Halo APU, `gfx1151`), 64 GiB BIOS carve-out seen as VRAM
  (+ 32 GB shared with system RAM; 108 GB usable by the GPU). No discrete PCIe link — CPU and iGPU share one pool.
- CPU: AMD Ryzen AI MAX+ PRO 395 w/ Radeon 8060S (16 cores / 32 threads, AVX-512)
- Installed RAM: 128 GB unified memory (64 GB seen by the OS + 64 GB VRAM carve-out)
- Storage: not yet verified; OS/SSD details not measured
- OS: Windows 11 (10.0.26200); GPU driver: 32.0.31036.15
- Strata: project 0.1.45 (git checkout `477ac13c`), engine **0.1.45 prebuilt** Windows HIP binary
  (`engine/BUILD.json`: prebuilt, windows-x64, archs incl. gfx1151, ROCm 10.2.0, hipBLASLt 100500), with the
  [#3098](https://github.com/Niko1221/Strata/pull/3098) bug fix used locally (not a source build of the engine)
- Background workloads and power limits: as in #3032, other programs were open during the suite; background load
  was not controlled and no power limit was set.

## Model and configuration

- Models (GGUF filenames from this install, quantization in brackets):
  - `qwen3.8-flash-next-iq3_xxs` [IQ3_XXS + PLE file] — `Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS-00001-of-00002.gguf`
    (+ `-00002` as `--ple-gguf`)
  - `qwen3.8-flash-next-unsloth-ud-iq4_xs` [UD-IQ4_XS] — `Qwen3.8-Flash-Next-UD-IQ4_XS-00001-of-00003.gguf` … `-00003`
  - `qwen3.6-35b-a3b-ud-q6_k_xl` [UD-Q6_K_XL, arch `qwen35moe`] — `Qwen3.6-35B-A3B-UD-Q6_K_XL.gguf` (native path,
    no pack/expert-profile; the file fits resident, no GPU expert cache)
- Context: 262144 tokens; KV type: int8; KV streaming on (setup default); expert cache: auto; vision: off.
- MTP: `rt` draft layer (`--spec 4`) for the Flash-Next packs; qwen3.6 native default `--mtp 2` (one variant `--mtp 4`).
  Thinking off, temperature 0 (suite standard). Calibration re-measured per model for this build
  (`START-HERE.bat --calibrate`); iq3_xxs settled at `--pcie-frac 1.00 --spec-min-p 0.70` (43.5 tok/s at calibration).
- Environment: `STRATA_HIPBLASLT_TUNING=gfx1151-hipblaslt-100500.txt`; variants additionally set
  `STRATA_SH_STREAM=0`, `STRATA_VERIFY_PROFILE=1`.

Representative launch commands (full per-variant command in each variant's `config-used.json` and `README.md`):

```text
# iq3_xxs variant "--mmap-experts STRATA_SH_STREAM=0"
strata.exe --pack iq3_xxs --native Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS-00001-of-00002.gguf \
  --ple-gguf Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS-00002-of-00002.gguf \
  --expert-profile expert-profile.bin --expert-cache auto --prefill auto \
  --spec 4 --mtp rt --max-context 262144 --kv int8 --pcie-frac 1.00 --spec-min-p 0.70 --mmap-experts
  env: STRATA_HIPBLASLT_TUNING=gfx1151-hipblaslt-100500.txt, STRATA_SH_STREAM=0

# unsloth-ud-iq4_xs default (START-HERE config; other variants add --pool-workers / --mmap-experts / --expert-cache)
strata.exe --pack unsloth-ud-iq4_xs --native Qwen3.8-Flash-Next-UD-IQ4_XS-00001-of-00003.gguf \
  --expert-profile expert-profile.bin --expert-cache auto --prefill auto \
  --spec 4 --mtp rt --max-context 262144 --kv int8 --kv-resident 32768 --resident-budget-gib 36 --spec-min-p 0.5

# qwen3.6-35b-a3b-ud-q6_k_xl default (native path)
strata.exe --native Qwen3.6-35B-A3B-UD-Q6_K_XL.gguf --max-context 262144 --prefill auto --mtp 2
```

## Method

Same self-written updated suite as #3032 (`suite/strata_suite.py` + `suite.toml` in the Strata folder): per model it runs
`START-HERE.bat` setup and `--calibrate`, edits the generated config per variant, then `BENCH.bat --long`
(`tools/strata_bench.py --long`): short chat, fresh 4K/16K/32K prompts (unique tag, no cache help, 24 new tokens),
5-turn agent session with the conversation cache. 3 runs per test, medians with ranges; engine timings for
prompt/decode, client-measured TTFT. 90 s cooldown between runs; stale engines killed first; all bench calls pass
`--config` and the same port (finding from [#3029](https://github.com/Niko1221/Strata/pull/3029)), so every variant
measures its own freshly started server. Suite window: 2026-10-19 22:09 → 2026-10-20 01:12.

Memory is now sampled every 5 s during each run (`memory_samples.csv` next to each variant's data):
`RAM strata` = strata.exe private working set, `RAM system` = total in-use system RAM, `VRAM` = dedicated GPU
memory, `shared GPU` = shared GPU memory (system RAM mapped for the GPU); the peaks include model load.

## Results

Ranked by chat decode. `pp` = prompt tok/s. Full ranking views `ranking.md` / `ranking.csv`, per-run folders
(the generated `README.md` + `results.json` + `engine-log-excerpt.txt`) and `comparisons/` keep the ranges.

| # | run | chat tok/s | agent tok/s | 4K pp | 16K pp | 32K pp | 32K TTFT s | RAM strata GiB | RAM system GiB | VRAM GiB | shared GPU GiB | expert cache GiB |
|---:|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | qwen36-ud-q6_k_xl:default | 50.5 | 38.0 | 578.8 | 482.9 | 373.5 | 85.8 | 30.6 | 46.7 | 49.4 | 0.8 | - |
| 2 | qwen36-ud-q6_k_xl:--mmap-experts STRATA_SH_STREAM=0 --mtp 4 | 50.5 | 38.1 | 595.8 | 479.3 | 379.2 | 84.6 | 30.6 | 46.4 | 49.4 | 0.8 | - |
| 3 | qwen36-ud-q6_k_xl:--mtp 4 | 50.5 | 37.9 | 591.8 | 471.5 | 375.1 | 85.5 | 30.6 | 46.4 | 49.4 | 0.8 | - |
| 4 | unsloth-ud-iq4_xs:--pool-workers 6 --mmap-experts STRATA_SH_STREAM=0 | 39.6 | 37.7 | 657.4 | 663.2 | 644.4 | 49.8 | 8.3 | 29.8 | 63.7 | 12.0 | 55.4 |
| 5 | iq3_xxs:--mmap-experts STRATA_SH_STREAM=0 | 39.4 | 44.7 | 620.3 | 636.0 | 620.3 | 51.7 | 41.5 | 57.5 | 50.5 | 0.9 | 40.0 |
| 6 | iq3_xxs:--pool-workers 6 STRATA_VERIFY_PROFILE=1 --mmap-experts STRATA_SH_STREAM=0 | 38.6 | 43.3 | 621.4 | 636.5 | 620.1 | 51.8 | 41.4 | 57.3 | 50.5 | 0.9 | 40.0 |
| 7 | unsloth-ud-iq4_xs:--mmap-experts STRATA_SH_STREAM=0 | 38.5 | 38.1 | 657.5 | 664.1 | 645.2 | 49.8 | 8.3 | 29.6 | 63.7 | 12.0 | 55.4 |
| 8 | iq3_xxs:--pool-workers 6 STRATA_VERIFY_PROFILE=1 | 36.2 | 40.4 | 565.9 | 605.2 | 590.1 | 54.4 | 41.4 | 57.3 | 26.6 | 40.8 | 16.0 |
| 9 | iq3_xxs:--mmap-experts | 34.7 | 40.0 | 612.6 | 637.1 | 618.6 | 51.9 | 41.6 | 57.5 | 50.7 | 0.9 | 40.0 |
| 10 | unsloth-ud-iq4_xs:--expert-cache 20000 | 34.5 | 32.7 | 656.3 | 664.7 | 645.3 | 49.8 | 8.3 | 29.9 | 63.7 | 12.2 | 55.4 |
| 11 | iq3_xxs:low-ram mmap | 34.3 | 38.7 | 604.4 | 616.3 | 621.4 | 51.6 | 41.6 | 58.4 | 50.7 | 0.9 | 40.0 |
| 12 | unsloth-ud-iq4_xs:default | 34.3 | 33.8 | 656.8 | 662.9 | 643.4 | 49.9 | 8.3 | 30.0 | 63.8 | 12.3 | 55.4 |
| 13 | unsloth-ud-iq4_xs:--pool-workers 6 | 34.1 | 33.8 | 655.8 | 663.8 | 641.9 | 50.0 | 8.3 | 29.8 | 63.8 | 12.4 | 55.4 |

One configuration did not complete: the first calibration attempt for qwen36-ud-q6_k_xl reported
"calibration: no result" (stored, no values); the three qwen3.6 variants themselves ran OK, without calibration
values applied.

### What was better than what (vs. each model's default / equivalent)

| Change | chat decode | agent decode | prompt tok/s | note |
|---|---|---|---|---|
| unsloth: + `--mmap-experts STRATA_SH_STREAM=0` | **+12.2 %** | **+12.7 %** | ±0 % | chat TTFT −5 % |
| unsloth: + `--pool-workers 6 --mmap-experts STRATA_SH_STREAM=0` | **+15.5 %** | +11.5 % | ±0 % | best unsloth chat |
| iq3_xxs: `--mmap-experts` with `STRATA_SH_STREAM=0` vs 1 | **+13.5 %** (39.4 vs 34.7) | +11.8 % | ±0 % | best iq3 chat + agent |
| unsloth: `--expert-cache 20000` | +0.6 % | −3.3 % | ±0 % | no effect (cache already fills VRAM) |
| qwen3.6: `--mtp 4` / `+--mmap-experts STRATA_SH_STREAM=0` | ±0 % | ±0 % | +2…+3 % at 4K | within noise |
| unsloth UD-IQ4_XS prompt speed, 0.44.1 → 0.1.45 | — | — | **~330 → ~655** | cross-suite, not a controlled A/B |

- `STRATA_VERIFY_PROFILE=1` / `--pool-workers 6` without the mmap changes is no longer a win on 0.1.45
  (iq3_xxs pool-workers variant is 2.8 tok/s *below* `--mmap-experts STRATA_SH_STREAM=0`).
- Docs impact: `STRIX_HALO.md` marks `STRATA_SH_STREAM=1` as the gfx1151 default because it paid on 0.1.40;
  on this 0.1.45 build the opposite measured (both models). A calibration fork "SH_STREAM off vs on" (as
  `tools/calibrate.py` already does for other engines) would decide per PC.

### Prompt speed is strongly model-dependent

- unsloth-ud-iq4_xs: 657 / 663 / 645 at 4K/16K/32K — flat, and ~2× its own 0.44.1 value from #3032
  (where it was the slow prefiler: 322–355 tok/s vs 543–615 for iq3_xxs then).
- iq3_xxs: 604–637 depending on variant — also flat.
- qwen3.6-35b-a3b-ud-q6_k_xl: 579–596 at 4K already the lowest, and it degrades with length:
  483 at 16K, 373 at 32K (TTFT 85 s for one 32K prompt). Its chat decode is the suite leader (50.5 tok/s),
  so this is where this model wants attention.
- No Task Manager re-read this time; the 5 s sampler (`memory_samples.csv`) replaces it — prefill pushed VRAM
  to ~63 / 50 / 49 GiB (unsloth / iq3_xxs / qwen3.6) with no visible disk activity for the mmap variants.

### qwen3.6: conversation cache not hitting

The same 5-turn agent session that the Flash-Next packs answer with a 68.6–68.8 % cache hit (4–73 tokens re-read)
runs on qwen3.6-35b-a3b-ud-q6_k_xl with **0 % reused, ~30,119 tokens re-read per session**; per-turn TTFT grows
1.2 → 5.7 → 8.4 → 14.7 → 20.2 → 26.8 s and agent decode sags to 30 tok/s at the last turn. Not measured whether
this is the `qwen35moe` native path, the missing pack, or KV handling — reporting the observation only.

## Possible improvement: shared GPU memory instead of normal RAM

The 0.45 sampler makes the placement visible. Two opposite styles on one memory pool:

| Variant (peak) | RAM strata | RAM system | VRAM | shared GPU |
|---|---:|---:|---:|---