Pull requests / #3163

#3163 hip: STRATA_SELECT_SPLIT2=1 - the selection scorer's hi/mid bf16 pieces (3 wmma instead of 6)

open · @vlupilin · 0 コメント · GitHub で見る

BenchmarksAMD / HIPNVIDIA / CUDAModels & quants

本文

## Summary

Opt-in on top of `STRATA_SELECT_WMMA=1` (AMD gfx11/gfx12 WMMA scorer). The `split3()` already writes each fp32 operand as three bf16 pieces (hi, mid, lo); this keeps the two largest (hi, mid) and drops the three products that carry the lo piece (each <= 2^-16 of the leading term: rel err ~1.5e-5, the class upstream's own WMMA-vs-warp gate accepts at 254/256).

Companion to the accepted wide top-k (#1749): together they optimize both stages of the QSA selection pipeline — the top-k that #1749 sped up, and the scorer this PR halves.

## Measurements (gfx1151, Radeon 8060S, Strix Halo)

**Kernel level** (`qsa_select_bench`, 256 queries):

| blocks | 6-product | split2 | speedup |
|---|---:|---:|---:|
| 8K | 1.08 ms | 0.53 ms | 2.0x |
| 65K | 12.5 ms | 6.6 ms | 1.9x |

**Selections identical**: 256/256 at 32K context, 253/256 at 262K (cells 0.0013%; upstream's own WMMA-vs-warp baseline is 254/256).

**End-to-end** (98K real prompt, interleaved x2, all arms with `STRATA_GDN_CHUNKED=1`):

| arm | tok/s | delta |
|---|---:|---:|
| mainline | 984.4 | — |
| +SPLIT2 | 1002.9 | **+1.9%** |

Also measured +1.3% against the four-lanes thread scorer (#1943) on the same card (interleaved T/S/T/S), and split2 covers the sub-16,385-block calls where the thread scorer never engages.

## Scope

- One file changed: `src/kernels/cuda/qsa_select.cu` (+35/-14)
- `STRATA_SELECT_SPLIT2=1` opt-in; default off = the 6-product kernel's bits unchanged
- CUDA does not compile this kernel (it's the HIP WMMA path)

関連リンク

インストール・モデル・リリースへの站内リンク。