#5025 [Engine] hip: gfx1151 hipBLASLt rows for Qwen3.6 prompt projections
open · @storm-ace · 0 comments · View on GitHub
BenchmarksAMD / HIPModels & quantsDocumentationWindowsLinux
Description
### Problem and result On a Strix Halo (gfx1151) with hipBLASLt 1.5.0, every Qwen3.6-35B-A3B prompt projection missed `tools/hip/gfx1151-hipblaslt-100500.txt` and ran on the hipBLAS fallback: `STRATA_HIPBLASLT_VERBOSE=1` reported `launches=0 fallbacks=1250` for a 4K prompt. The table only had rows for the Flash-Next shapes. This adds 35 rows for Qwen3.6's shapes. With them, the prompt read on a Radeon 8060S goes from ~590 to ~1,100 tok/s at 4K and from ~350 to ~500 tok/s at 32K (`fallbacks=0`). Decode does not change. Data only: no code changes, and the existing 32 rows are untouched. ### Related discussion None needed for a table addition, as far as I can tell. #4295 adds a gfx1151 table for hipBLASLt 1.2.2 (system ROCm 7.2.4); this one is for the 1.5.0 table the Windows build ships. ### Approach - Shapes: hidden 2048, so N x K = 8192 x 2048, 4096 x 2048, 2048 x 4096, 512 x 2048, 2048 x 512 (the projections `prefill::Gemm::native` runs for this model), at T = 16, 64, 256, 512, 1024, 2048, 4096. T up to 4096 covers the prompt chunk the engine picks with every expert in VRAM. - Calibrated with `tune_hipblaslt` on the same card and ROCm as the measurements below. The tuner picked solution 2540, 2541 or 2542 for every row. For T = 4096, N = 2048, K = 512 it reported `speedup_vs_blas=5.5822x` (the one row whose tuner output I kept); the whole-model numbers below are the claim. - `docs/AMD_HIP.md`: the 100500 file's note and a row in the measured table. ### Validation Environment: Ryzen AI MAX+ PRO 395, Radeon 8060S (gfx1151), Windows 11 Pro 10.0.26200, built from this branch (upstream `c62d5e3a` + this table) with `tools/hip/build_windows.bat` (`STRATA_HIP_ARCHS=gfx1151`, ROCm 10.2.0a20260930 TheRock wheels, hipBLASLt 1.5.0), Qwen3.6-35B-A3B UD-Q4_K_XL, every routed expert in VRAM, `--max-context 131072`, prompt chunk 4096 (auto). **Passed** - A/B with the same binary. The only difference between arms is `STRATA_HIPBLASLT_TABLE=off`. Arms in the order off / on / off, each a fresh engine start: `strata --native <gguf> --max-context 131072 --bench-pp N --bench-tg 32`, 3 repetitions per arm: | Prompt | Table off | Table on | Table off again | | --- | --- | --- | --- | | pp4096 | 588.3 / 577.1 / 594.3 tok/s | 1,147.1 / 1,093.2 / 1,106.9 | 592.5 / 591.9 / 595.4 | | pp32768 | 366.0 / 345.3 / 352.3 | 500.5 / 502.4 / 495.6 | 345.9 / 352.1 / 343.6 | | tg32 after pp4096 | 42.5 / 42.2 / 42.5 | 42.5 / 42.9 / 42.4 | 42.3 / 42.8 / 41.7 | | tg32 after pp32768 | 12.5 / 12.5 / 13.4 | 12.5 / 12.6 / 13.4 | 12.6 / 12.6 / 12.6 | With the table: `prefill gemm: hipBLASLt tuning enabled (67 rows, gfx1151, version 100500)` and `hipBLASLt summary launches=1250 fallbacks=0` (pp4096) / `launches=6500 fallbacks=0` (pp32768). - Output: teacher-forced logits, table off against on, same binary. 1,229-token real-text prompt (`tools/qwen35/parity_prompts.ids` row 1 plus its greedy continuation), 32 forced steps, `--dump-full-logits`. In 256-cell chunks (`STRATA_Q36_CHUNK=256`): mean KL 1.41e-6, max 1.10e-5, top-1 32 of 32. With chunk 4096: mean KL 1.50e-6, max 1.39e-5, top-1 32 of 32. (For scale, the floor in docs/QWEN36.md, llama.cpp CPU against HIP, is 0.0019 to 0.056.) **Failed:** none observed. **Not run** - Flash-Next and DeepSeek on this branch. Their shapes have different N / K, so these rows do not match them, and their rows are unchanged; but I did not re-measure them. - Other gfx1151 machines, Linux, and hipBLASLt versions other than 1.5.0 (`100500`). The file is version-scoped as before. - No GPU build beyond HIP gfx1151, since no code changed. ### Limits - The rows are for this hipBLASLt build. The engine checks only the version number, as noted for the other 1.5.0 tables. - Other AMD tables (gfx1100, gfx1150, gfx1200, gfx1201) have no Qwen3.6 rows either. Owners of those cards could calibrate the same 35 cases with `tune_hipblaslt`; I have only this card. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Related on this site
Quick links to related pages.