#4295 [Engine] gfx1151 hipBLASLt table for a system ROCm 7.2.4 (hipBLASLt 100202)
open · @AMV007 · 0 Kommentare · Auf GitHub
BenchmarksSetup & installMulti-GPUAMD / HIPModels & quantsDocumentation
Beschreibung
## Problem and result On Strix Halo with a system ROCm 7.2.4 in /opt/rocm (installed for llama.cpp), setup builds the engine against it rather than the TheRock wheels, because `rocm_root` prefers a system ROCm 7+. `tools/hip` had no gfx1151 table for its hipBLASLt 1.2.2, so the prompt's dense GEMMs ran on plain hipBLAS. After a `git pull` rebuild, big prompts were about 2x slower than with the 7.14 wheels. Before: UD-Q4_K_XL 59K prompt 247 tok/s, 232K 228 tok/s. After: 471 tok/s on 59K, on par with the 7.14 wheels + 100400 table (461). ## Related discussion none ## Approach Adds `tools/hip/gfx1151-hipblaslt-100202.txt`, tuned with `tools/hip/tune_hipblaslt` against /opt/rocm. It covers the 15 shapes and 6 token buckets of the 100401 table (90 rows). `hipblaslt_table()` picks it when the installed hipBLASLt is 100202; nothing changes for other versions. Also included: docs rows in AMD_HIP.md, a report in `bench/results/2026-10-27-community-gfx1151-hipblaslt-100202/` (tune command, shapes, numbers), and a check in `test_setup_strix_halo`. As with every table, the prompt GEMMs use other kernels, so prompt numerics differ from plain hipBLAS in the last bits. The tuner's accuracy gate is rel-L2 1e-4; the rows here have about 2e-7. This matches the 7.14 wheels, which already use a table by default on Strix Halo. ## Validation **Passed:** - `LD_LIBRARY_PATH=/opt/rocm/lib build-hip/tune_hipblaslt --shapes-file shapes.json --tokens 2047,2048,8191,8192,16383,16384 --workspace-mib 32 --tuning-out gfx1151-hipblaslt-100202.txt` on 4b9e05a2: meta `device_arch=gfx1151 hipblaslt_version=100202`, 90 cases, 0 rejected. GEMM sums, hipBLAS -> table: T 2048 115.5 -> 19.1 ms, T 8192 467.9 -> 77.2 ms, T 16384 945.4 -> 157.3 ms. - End to end on Ryzen AI Max+ 395, 128 GB, Ubuntu, kernel 7.0.0-38, ROCm 7.2.4, engine 0.1.51 (4b9e05a2), UD-Q4_K_XL, setup's config (`--resident-budget-gib 71`, `--max-context 262144`), fresh server, caches dropped, one run each: - 59K prompt: 247 -> 471 tok/s. - With the STRIX_HALO.md opt-in prompt set: 808 -> 1,107 on 59K, and 232K 1,034. - Decode unchanged (26-30 tok/s). - Log: `prefill gemm: hipBLASLt tuning enabled (90 rows, gfx1151, version 100202)`. - `python -m unittest tools.test_setup_strix_halo.Recommendation.test_hipblaslt_table_for_gfx1151 tests/terminal/test_hipblaslt_tuning.py`: OK. **Failed:** none observed **Not run:** - KL against plain hipBLAS or the 7.14 table. - Other gfx1151 boxes and other models; one run per point. ## Limits - With this table `STRATA_HCD_EXACT` has no solution 1176/1177 to match (hipBLASLt picks 6873) and says so in the log; hipBLASLt runs that projection. - The table is valid only for hipBLASLt 1.2.2.
Mehr auf der Site
Links zu Install, Modellen, Releases.