Pull requests / #2132

#2132 Community benchmark: Radeon AI PRO R9700 (gfx1201), ROCm 7.14.1, Ryzen 9 3900 (AVX2) — UD-IQ4_XS and the opt-in HIP switches, one at a time

open · @i-Vladimir-i · 0 comentários · No GitHub

BenchmarksAMD / HIPModels & quantsDocumentationWindows

Descrição

Results-only submission per docs/COMMUNITY_BENCHMARKS.md (no code change).

**Hardware:** one Radeon AI PRO R9700 32 GB (gfx1201), CPU-attached PCIe 4.0 x16; Ryzen 9 3900 (Zen 2, 12C/24T, AVX2, no AVX-512); 125 GiB DDR4; NVMe; Ubuntu 24.04, kernel 6.8, system ROCm 7.14.1 (hipBLASLt 1.4.1). Engine 0.1.43 (`7695859`) built from source for gfx1201; HIP ctest 104/105 (`ple_parity` needs the Q2_0 fixture).

**Model/config:** Unsloth UD-IQ4_XS (pinned SHA-256), all experts in RAM (`--resident-budget-gib 55`), expert cache auto, 262K context, `--kv int8 --kv-resident 32768`, MTP `--spec 4`.

**What it adds to the existing R9700 reports (#2047, #1580):** an A/B of each opt-in switch on one machine (`strata_bench --long`, 3 runs per arm, one switch per arm):
- `STRATA_HIP_WMMA=1`: prompts +25% (4K) to +50% (32K), agent TTFT −20%;
- `STRATA_ROUTE_RESIDENT=1`: decode +11% (chat) / +7% (agent) — larger than the R9700 figure in the merge note; this CPU is Zen 2 with AVX2 only;
- `STRATA_PF_FUSED=1`: no change with this (Unsloth) pack;
- a hipBLASLt **1.4.1** table calibrated on the 15 dense shapes the model actually logs (seed table + `STRATA_HIPBLASLT_VERBOSE=1`): 0.96–3.06× per GEMM, end to end within noise — confirms the AMD_HIP.md note; table included as data;
- `--prefill 8192`: 6–8% slower than auto (16,384); the lent-slot "read from the SSD" warning did not show as disk reads with this much RAM;
- `"parallel": 2` vs 1 with two clients: the second slot gives an earlier first token but a lower total and later finishes.

Not tested: quality benchmarks, prompts beyond 32K, Windows, the GSQ-RCO packs on this ROCm (IQ3_S is the next run).

No site

Links install, modelos, releases.