Pull requests / #2276

#2276 bench: community report, RX 9060 XT (gfx1200) with 16 GB of RAM and a SATA SSD, Coder IQ1_M at 128K on 0.1.43.1, stock vs tuned

open · @mandoof1 · 0 comentários · No GitHub

BenchmarksSetup & installAMD / HIPNVIDIA / CUDAModels & quantsDocumentationWindowsLinux

Descrição

Issue: none (results-only report)

## Summary
The Coder IQ1_M runs on a Linux PC with 15.5 GiB of usable RAM, half of the 32 GB the README lists for it, with the experts read from a SATA SSD (`--mmap-experts`). This report gives a same-day comparison, on 0.1.43.1 (`a0116b2`), between stock with setup's own configuration and the configuration I use every day, and lists what made the difference.

| Same day, same PC | Stock, setup's config (16K) | Tuned (128K) |
| --- | --- | --- |
| short chat decode | 2.6 tok/s, 2 of 9 requests ended the engine | 8.9 tok/s, no failures |
| 4K prompt | 43.6 tok/s, first token 92 s | 178.4 tok/s, first token 23 s |
| 16K / 32K prompt | do not fit | 323 / 373 tok/s |
| agent session (6 turns x 3) | first turn: no output for 312 s, engine ended; stopped | 18/18 turns, 68.7% cache hit |

The 8.9 tok/s is the bench's short chat (100-token replies, thinking off). In everyday agent use with thinking on, decode runs at 12-17 tok/s with peaks of 25 (earlier measurements in the report).

The stock run shows `verify: timed out at layer 19` (#267) and 3.4 M major faults in one request: the experts arrive about 4 KB at a time from the SATA drive.

The settings that mattered most here:
- `GPU_PINNED_MIN_XFER_SIZE=1048576` (the ROCclr USERPTR mechanism from TROUBLESHOOTING.md / #1705; upstream sets it only with `STRATA_ARENA_MMAP=1`, not on the `--mmap-experts` path)
- the server in a `MemorySwapMax=0` systemd scope (zram is on by default on CachyOS)
- static residency with `--adapt-every 100000`
- longer stall limits
- an expert profile learned from agent prompts (included, 98 KB)

## What changed
Only a new folder, `bench/results/2026-10-13-community-rx-9060-xt-16gb-ram/`:
- `README.md`: hardware, configuration, method, results, earlier tuning measurements, and the settings that were measured slower here
- `tuned-128k/`, `stock-16k/`: the `strata_bench.py` output of each run (`README.md`, `results.json`, `engine-log-excerpt.txt`) and a 2 s RAM/VRAM/disk sample file (`mem-samples.txt`)
- `strata-coder-iq1_m-128k.json`: the tuned configuration, paths replaced
- `expert-profile-agent.bin`: the agent-learned expert profile for the Coder IQ1_M pack
- `local-patches.diff`: three small local engine patches used in the tuned run, for reproduction only

No source, docs or defaults are changed.

## Extra Notes
- The local patches are a `STRATA_VERIFY_TIMEOUT_S` env, `MADV_WILLNEED` hints before prefill staging and the post-prompt slot refill, and `STRATA_PROFILE_ROUTING_ONLY=1` for `--expert-profile-save` under static residency. They are built and tested with HIP only. If any of them is wanted as an engine change, I can open it as its own PR with the measurements and builds it needs.
- Not tested: CUDA and SYCL builds of the patches, Windows, NVMe, thinking-on speed through the suite, correctness checks, more than one slot. Each configuration was run once, tuned first.

No site

Links install, modelos, releases.