贡献 / #2276
#2276 bench: community report, RX 9060 XT (gfx1200) with 16 GB of RAM and a SATA SSD, Coder IQ1_M at 128K on 0.1.43.1, stock vs tuned
open · @mandoof1 · 0 评论 · 去 GitHub 看
BenchmarksSetup & installAMD / HIPNVIDIA / CUDAModels & quantsDocumentationWindowsLinux
说明
Issue: none (results-only report) ## Summary The Coder IQ1_M runs on a Linux PC with 15.5 GiB of usable RAM, half of the 32 GB the README lists for it, with the experts read from a SATA SSD (`--mmap-experts`). This report gives a same-day comparison, on 0.1.43.1 (`a0116b2`), between stock with setup's own configuration and the configuration I use every day, and lists what made the difference. | Same day, same PC | Stock, setup's config (16K) | Tuned (128K) | | --- | --- | --- | | short chat decode | 2.6 tok/s, 2 of 9 requests ended the engine | 8.9 tok/s, no failures | | 4K prompt | 43.6 tok/s, first token 92 s | 178.4 tok/s, first token 23 s | | 16K / 32K prompt | do not fit | 323 / 373 tok/s | | agent session (6 turns x 3) | first turn: no output for 312 s, engine ended; stopped | 18/18 turns, 68.7% cache hit | The 8.9 tok/s is the bench's short chat (100-token replies, thinking off). In everyday agent use with thinking on, decode runs at 12-17 tok/s with peaks of 25 (earlier measurements in the report). The stock run shows `verify: timed out at layer 19` (#267) and 3.4 M major faults in one request: the experts arrive about 4 KB at a time from the SATA drive. The settings that mattered most here: - `GPU_PINNED_MIN_XFER_SIZE=1048576` (the ROCclr USERPTR mechanism from TROUBLESHOOTING.md / #1705; upstream sets it only with `STRATA_ARENA_MMAP=1`, not on the `--mmap-experts` path) - the server in a `MemorySwapMax=0` systemd scope (zram is on by default on CachyOS) - static residency with `--adapt-every 100000` - longer stall limits - an expert profile learned from agent prompts (included, 98 KB) ## What changed Only a new folder, `bench/results/2026-10-13-community-rx-9060-xt-16gb-ram/`: - `README.md`: hardware, configuration, method, results, earlier tuning measurements, and the settings that were measured slower here - `tuned-128k/`, `stock-16k/`: the `strata_bench.py` output of each run (`README.md`, `results.json`, `engine-log-excerpt.txt`) and a 2 s RAM/VRAM/disk sample file (`mem-samples.txt`) - `strata-coder-iq1_m-128k.json`: the tuned configuration, paths replaced - `expert-profile-agent.bin`: the agent-learned expert profile for the Coder IQ1_M pack - `local-patches.diff`: three small local engine patches used in the tuned run, for reproduction only No source, docs or defaults are changed. ## Extra Notes - The local patches are a `STRATA_VERIFY_TIMEOUT_S` env, `MADV_WILLNEED` hints before prefill staging and the post-prompt slot refill, and `STRATA_PROFILE_ROUTING_ONLY=1` for `--expert-profile-save` under static residency. They are built and tested with HIP only. If any of them is wanted as an engine change, I can open it as its own PR with the measurements and builds it needs. - Not tested: CUDA and SYCL builds of the patches, Windows, NVMe, thinking-on speed through the suite, correctness checks, more than one slot. Each configuration was run once, tuned first.
本站相关内容
相关页面的快捷入口。