#4295 [Engine] gfx1151 hipBLASLt table for a system ROCm 7.2.4 (hipBLASLt 100202)

open · @AMV007 · 0 comments · View on GitHub

BenchmarksSetup & installMulti-GPUAMD / HIPModels & quantsDocumentation

Description

## Problem and result
On Strix Halo with a system ROCm 7.2.4 in /opt/rocm (installed for llama.cpp), setup builds the engine against it rather than the TheRock wheels, because `rocm_root` prefers a system ROCm 7+. `tools/hip` had no gfx1151 table for its hipBLASLt 1.2.2, so the prompt's dense GEMMs ran on plain hipBLAS. After a `git pull` rebuild, big prompts were about 2x slower than with the 7.14 wheels.

Before: UD-Q4_K_XL 59K prompt 247 tok/s, 232K 228 tok/s. After: 471 tok/s on 59K, on par with the 7.14 wheels + 100400 table (461).

## Related discussion
none

## Approach
Adds `tools/hip/gfx1151-hipblaslt-100202.txt`, tuned with `tools/hip/tune_hipblaslt` against /opt/rocm. It covers the 15 shapes and 6 token buckets of the 100401 table (90 rows). `hipblaslt_table()` picks it when the installed hipBLASLt is 100202; nothing changes for other versions. Also included: docs rows in AMD_HIP.md, a report in `bench/results/2026-10-27-community-gfx1151-hipblaslt-100202/` (tune command, shapes, numbers), and a check in `test_setup_strix_halo`.

As with every table, the prompt GEMMs use other kernels, so prompt numerics differ from plain hipBLAS in the last bits. The tuner's accuracy gate is rel-L2 1e-4; the rows here have about 2e-7. This matches the 7.14 wheels, which already use a table by default on Strix Halo.

## Validation
**Passed:**
- `LD_LIBRARY_PATH=/opt/rocm/lib build-hip/tune_hipblaslt --shapes-file shapes.json --tokens 2047,2048,8191,8192,16383,16384 --workspace-mib 32 --tuning-out gfx1151-hipblaslt-100202.txt` on 4b9e05a2: meta `device_arch=gfx1151 hipblaslt_version=100202`, 90 cases, 0 rejected. GEMM sums, hipBLAS -> table: T 2048 115.5 -> 19.1 ms, T 8192 467.9 -> 77.2 ms, T 16384 945.4 -> 157.3 ms.
- End to end on Ryzen AI Max+ 395, 128 GB, Ubuntu, kernel 7.0.0-38, ROCm 7.2.4, engine 0.1.51 (4b9e05a2), UD-Q4_K_XL, setup's config (`--resident-budget-gib 71`, `--max-context 262144`), fresh server, caches dropped, one run each:
  - 59K prompt: 247 -> 471 tok/s.
  - With the STRIX_HALO.md opt-in prompt set: 808 -> 1,107 on 59K, and 232K 1,034.
  - Decode unchanged (26-30 tok/s).
  - Log: `prefill gemm: hipBLASLt tuning enabled (90 rows, gfx1151, version 100202)`.
- `python -m unittest tools.test_setup_strix_halo.Recommendation.test_hipblaslt_table_for_gfx1151 tests/terminal/test_hipblaslt_tuning.py`: OK.

**Failed:**
none observed

**Not run:**
- KL against plain hipBLAS or the 7.14 table.
- Other gfx1151 boxes and other models; one run per point.

## Limits
- With this table `STRATA_HCD_EXACT` has no solution 1176/1177 to match (hipBLASLt picks 6873) and says so in the log; hipBLASLt runs that projection.
- The table is valid only for hipBLASLt 1.2.2.

Related on this site

Quick links to related pages.