#3032 community test suite reports, Radeon 8060S (gfx1151) on a Ryzen AI Max+ 395 (Windows 11)

Descrição

# Community benchmark on AMD Radeon 8060S (Strix Halo, gfx1151)

Measured on 2026-10-16 / 2026-10-17 by `mistral13`. This is a **configuration-comparison suite**: the same machine
runs three quantizations of Qwen3.8-Flash-Next (IQ3_XXS, Unsloth UD-IQ4_XS, Coder IQ1_M) across many engine
variants, and the variants are ranked against each other. It is a **speed-only** suite — no correctness or recall
check was run. The main limitation is that other programs were running on the machine during the suite (the exact
background load was not recorded), and the storage type and the ROCm/driver version were not captured.

## Hardware and software

- GPU and VRAM: AMD Radeon(TM) 8060S Graphics (Strix Halo APU, `gfx1151`), unified memory: 64.0 GB BIOS carve-out
  + 32 GB shared with system RAM, **108 GB usable by the GPU**. There is no discrete PCIe GPU link — the CPU and the
  iGPU share one memory pool.
- CPU: AMD Ryzen AI MAX+ PRO 395 w/ Radeon 8060S (16 cores / 32 threads, AVX-512)
- Installed RAM: 128 GB  unified memory (64GB RAM; 64GB VRAM)
- Storage: not yet verified (model files are read from disk for the mmap / SSD-backed variants)
- PCIe link: not applicable (APU, unified memory)
- OS: Windows 11 (10.0.26200)
- Driver / ROCm: not recorded (HIP backend; the prebuilt gfx1151 AMD engine was used)
- Strata: release **0.44.1** (engine 0.1.44.1), **prebuilt release binary** (`engine/strata.exe`, llama.cpp `3cf0325`),
  not a source build
- Background workloads and power limits: other programs were running during the suite; no power limit was set on the
  APU. Background load was not otherwise controlled.

> Version note: the separate report `bench/2026-10-14-community-cpu-qwen3-8-flash-next-unsloth-ud-iq4-xs-2` was made
> with Strata **0.43.0**. Everything in this folder (`2026-10-16_2310`) was made with **0.44.1**.

## Model and configuration

- Models and quantization (GGUF filenames, no repository/revision recorded):
  - `qwen3.8-flash-next-iq3_xxs` — `Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS-00001-of-00002.gguf` + `-00002-of-00002.gguf`
    (the second file is loaded as `--ple-gguf`)
  - `qwen3.8-flash-next-unsloth-ud-iq4_xs` — `Qwen3.8-Flash-Next-UD-IQ4_XS-00001-of-00003.gguf` … `-00003-of-00003.gguf`
  - `qwen3.8-flash-next-coder-iq1_m` — Coder IQ1_M pack
- Vision encoder: off (images disabled). Custom profile: `data/expert-profile.bin` (expert-use profile).
- Context: 262144 tokens (256K). KV type: `int8`. KV streaming: on by default (KV cache lives in RAM); the
  `kv-streaming off` variants turn it off. Expert cache: `auto`, sized from the memory the OS can give back.
- Low-RAM mode: `auto` by default; variants use `--low-ram resident` (experts the GPU cannot hold are copied into RAM
  once) or `--low-ram mmap` (experts read through the OS file cache). Unsloth default keeps ~36 GiB of experts in RAM.
- MTP: draft layer `rt` (`--mtp rt`), `--spec 4`. Reasoning/thinking: **off** for the suite. Sampling: temperature 0.
- Calibration: run per model (`START-HERE.bat --calibrate`), which sets `--pcie-frac`, `--spec-min-p`,
  `--adapt-every`, `--adapt-swaps`, `--adapt-decay`. Experimental speed projection: **not used**.

Exact launch commands (paths shortened to the Strata folder; no credentials). Each variant edits these; the exact
per-variant command is in that variant's `config-used.json` and its own `README.md`.

```text
# iq3_xxs — install default (strata-iq3_xxs.json, as START-HERE left it)
strata.exe --pack packs/iq3_xxs \
  --native models/IQ3_XXS/Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS-00001-of-00002.gguf \
  --ple-gguf models/IQ3_XXS/Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS-00002-of-00002.gguf \
  --expert-profile data/expert-profile.bin --expert-cache auto --prefill auto \
  --spec 4 --spec-min-p 0.5 --mtp mtp/rt --max-context 262144 --kv int8
  env: STRATA_HIPBLASLT_TUNING=tools/hip/gfx1151-hipblaslt-100500.txt

# unsloth-ud-iq4_xs — install default
strata.exe --pack packs/unsloth-ud-iq4_xs \
  --native models/unsloth-UD-IQ4_XS/Qwen3.8-Flash-Next-UD-IQ4_XS-00001-of-00003.gguf \
  --expert-profile data/expert-profile.bin --expert-cache auto --prefill auto \
  --spec 4 --mtp mtp/rt --max-context 262144 --kv int8 --kv-resident 32768 \
  --resident-budget-gib 36 --spec-min-p 0.5
  env: STRATA_HIPBLASLT_TUNING=tools/hip/gfx1151-hipblaslt-100500.txt

# example variant: iq3_xxs --pool-workers 6 STRATA_VERIFY_PROFILE=1 --mmap-experts (low-ram mmap)
  ... --pcie-frac 1.00 --spec-min-p 0.70 --adapt-every 1 --adapt-swaps 80 --adapt-decay 0.97 \
  --pool-workers 6 --mmap-experts
  env: STRATA_HIPBLASLT_TUNING=tools/hip/gfx1151-hipblaslt-100500.txt, STRATA_VERIFY_PROFILE=1
```

## Method

Driven by a test suite I wrote (`suite/`, `suite/strata_suite.py` + `suite/suite.toml`), which is included with this
report. For every model the suite runs `START-HERE.bat` (setup, `--no-start`), then `START-HERE.bat --calibrate`
(per-model tuning, 15–30 min), then `BENCH.bat --long` against the HTTP API of a Strata server started from that
variant's config. Model loading is not timed. There is a 90 s cooldown between runs, and a stale engine is killed
before each step.

`BENCH.bat` runs `tools/strata_bench.py --long`: 3 runs per test, medians with ranges. Speeds are the engine's own
timings (`prompt_per_second`, `predicted_per_second`); first-token time and latency are measured by the client over
HTTP. One warm-up chat and a 1K prompt are discarded first. Every prompt carries a unique tag, so the fresh-prompt
tests never get help from the conversation cache.

- short chat: three questions, up to 100 new tokens, 3 runs
- 4K / 16K / 32K prompts: fresh word-list prompts (~4,096 / 16,384 / 32,768 tokens), 24 new tokens, 3 fresh prompts each
- agent session: system prompt + four tools, then 5 turns that each add ~1,500 words of a project file and continue
  from the model's own earlier answers (how agent clients resend a conversation), 64 new tokens per turn, 3 sessions;
  cache numbers come from the answer's usage (`cached_tokens`) and timings (`prompt_n`)

Peak RAM and peak GPU memory are sampled by the suite during the run (Windows only, includes model load): `peak_ram_gib`
is process RAM, `peak VRAM` is the GPU's dedicated memory, `expert cache GiB` is the GPU expert-cache size. The
memory/utilization notes below are separate Task Manager readings taken by hand during representative runs.

## Results

Ranked by chat decode (primary). `*` = variant that changes numerics (output not bit-identical — quality not verified).
Per-run detail (actual prompt tokens, reused tokens, generated tokens, runs, medians and ranges, TTFT, agent cache)
is in each variant folder's `README.md` and `results.json`; engine start-up lines are in `engine-log-excerpt.txt`.

| # | run | chat tok/s | agent tok/s | 4K pp | 16K pp | 32K pp | 32K TTFT s | peak RAM GiB | peak VRAM GiB | expert cache GiB |
|---:|---|---|---|---|---|---|---|---|---|---|
| 1 | iq3_xxs:--pool-workers 6 STRATA_VERIFY_PROFILE=1 --mmap-experts low-ram mmap | 38.2 | 42.7 | 548.1 | 565.2 | 550.9 | 58.3 | 41.5 | 53.1 | 40.0 |
| 2 | iq3_xxs:--pool-workers 6 STRATA_VERIFY_PROFILE=1 --mmap-experts | 37.7 | 42.3 | 548.5 | 566.9 | 547.6 | 58.6 | 41.5 | 53.1 | 40.0 |
| 3 | iq3_xxs:--mmap-experts | 35.5 | 37.5 | 593.0 | 615.1 | 549.7 | 58.4 | 41.6 | 53.3 | 40.0 |
| 4 | iq3_xxs:--pool-workers 6 STRATA_VERIFY_PROFILE=1 | 35.5 | 40.8 | 566.5 | 597.2 | 579.2 | 55.4 | 41.5 | 29.2 | 16.0 |
| 5 | iq3_xxs:--pool-workers 6 STRATA_VERIFY_PROFILE=1 host-core last | 35.4 | 60.9 | 552.6 | 571.1 | 558.0 | 57.5 | 41.4 | 29.2 | 16.0 |
| 6 | unsloth-ud-iq4_xs:--pool-workers 6 | 33.3 | 32.2 | 323.9 | 348.7 | 339.4 | 94.5 | 12.5 | 62.3 | 55.4 |
| 7 | iq3_xxs:low-ram mmap | 33.1 | 35.6 | 544.7 | 565.9 | 551.2 | 58.2 | 41.5 | 53.3 | 40.0 |
| 8 | iq3_xxs:--mmap-experts mmap | 33.1 | 33.7 | 543.1 | 561.5 | 546.1 | 58.8 | 41.5 | 53.3 | 40.0 |
| 9 | iq3_xxs:low-ram resident | 32.9 | 36.9 | 549.4 | 564.6 | 549.4 | 58.4 | 4.1 | 53.3 | 40.0 |
| 10 | iq3_xxs:low-ram mmap host-core last | 32.7 | 35.4 | 552.2 | 573.9 | 558.5 | 57.5 | 41.5 | 53.3 | 40.0 |
| 11 | unsloth-ud-iq4_xs:--pool-workers 6 STRATA_VERIFY_PROFILE=1 | 32.4 | 37.7 | 348.7 | 354.9 | 343.2 | 93.5 | 12.5 | 62.1 | 55.4 |
| 12 | unsloth-ud-iq4_xs:resident-budget-gib 36 | 32.0 | 34.7 | 318.0 | 351.4 | 340.5 | 94.2 | 12.5 | 62.3 | 55.4 |
| 13 | unsloth-ud-iq4_xs:default | 31.9 | 33.5 | 321.7 | 329.9 | 322.1 | 99.6 | 12.5 | 62.3 | 55.4 |
| 14 | iq3_xxs:low-ram mmap and kv-streaming off | 31.8 | 36.6 | 544.5 | 566.7 | 550.0 | 58.4 | 41.6 | 53.3 | 40.0 |
| 15 | coder-iq1_m:strix-prefill-switches * | 31.6 | 35.4 | 842.3 | 940.1 | 949.8 | 33.8 | 28.4 | 34.3 | 23.4 |
| 16 | coder-iq1_m:default | 29.1 | 33.1 | 654.1 | 654.0 | 632.0 | 50.8 | 28.5 | 34.7 | 23.4 |
| 17 | coder-iq1_m:low-ram mmap and kv-streaming off | 28.8 | 27.8 | 575.5 | 579.9 | 563.2 | 57.0 | 25.1 | 37.7 | 23.4 |

`pp` = prompt tok/s at that prompt length. Full ranking views (by 16K prompt, by agent decode, by peak RAM) are in
`ranking.md` / `ranking.csv`; variant-vs-default diffs are in `comparisons/`.

### Memory and utilization observations (Task Manager, hand-read during representative runs)

These are my own readings, not engine timings. "Shared GPU Memory" is the system-memory-backed GPU allocation,
"VRAM" is dedicated GPU memory. CPU figures are per-core load during prefill vs generation.

| Config | RAM used | Shared GPU mem | VRAM | Prefill (CPU / GPU / disk) | Generate (CPU / GPU / disk) |
|---|---|---|---|---|---|
| iq3_xxs default (install config) | ~80% | 41 GB | 26 GB | core 0 100%, small use on 2–7 & 10–15 / ~90% / none | even cores 0,2,…,30 100%, odd cores 1,3,…,31 0% / ~90% / none |
| iq3_xxs --pool-workers 6 STRATA_VERIFY_PROFILE=1 --mmap-experts | ~80% | 1 GB | 50 GB | core 0 100%, peaks 2–7 & 10–15, rest 0% / ~95% / none | core 0 100%, peaks 2–7 & 10–15, rest 0% / ~95% / none |
| iq3_xxs --pool-workers 6 STRATA_VERIFY_PROFILE=1 host-core last | ~80% | 40 GB | 27 GB | core 0 0%, some on 1, peaks 2–7 & 10–15, rest 0% / ~95% / none | peaks on 0–15, rest 0% / ~98% / none |
| iq3_xxs low-ram mmap | ~80% | 1 GB | 51 GB | core 0 100%, higher peaks 2–7 & 10–15, rest 0% / ~98% / none | core 0 100%, peaks 2–7 & 10–15, rest 0% / ~80% / small |
| unsloth-ud-iq4_xs default | ~10 GB | 10 GB | 63 GB | core 0 100%, all 1–31 100% with short 0% dips (incl. hyperthreads) / ~98% / small | core 0 100%, peaks 2–15 (and 2–7 & 10–15), rest 0% / ~80–85% / small |

Pattern seen: the mmap / `--mmap-experts` variants push experts into dedicated VRAM (low shared memory, high VRAM) and
decode almost single-threaded on core 0; the resident / `host-core last` variants keep the expert cache in shared
memory (high shared, low VRAM) and spread load across cores. Unsloth UD-IQ4_XS fills VRAM (~62–63 GB) and uses little
process RAM (~10–12 GiB).

### Findings

- Fastest chat decode: `iq3_xxs` with `--pool-workers 6 STRATA_VERIFY_PROFILE=1 --mmap-experts` (38.2 tok/s chat,
  42.7 tok/s agent).
- Best agent decode: `iq3_xxs ... host-core last` (60.9 tok/s) — moving the host core to the last core helps the
  agent (cache-heavy, multi-turn) workload most.
- Fastest prompt ingestion by far: `coder-iq1_m strix-prefill-switches` (842 / 940 / 950 tok/s at 4K/16K/32K, 32K
  TTFT 33.8 s) — but it changes numerics, so quality is unverified.
- Lowest process RAM: `iq3_xxs low-ram resident` (4.1 GiB) — experts copied into RAM once, no mmap churn.
- Unsloth UD-IQ4_XS is the slowest to prompt (322–355 tok/s, 32K TTFT ~94–100 s) because most experts come from disk.

## Suite folder

I built a repeatable test suite in `suite/` (`strata_suite.py`, `suite.toml`, plus `RUN_SUITE.bat`,
`CONFIGS_SUITE.bat`, `RANK_SUITE.bat`, `COMPARE_SUITE.bat`) and I will add it with this report. It runs the
setup → calibrate → bench sequence for every model and variant above, writes the per-run folders, the `ranking.*`
tables and the `comparisons/` diffs, and compares each variant against the model's default config.

## Correctness and li