改动 / #3164

#3164 hip: STRATA_SELECT_GRID=2 - the selection scorer's CTA order swaps for L2 (bitwise identical)

open · @vlupilin · 0 评论 · 去 GitHub 看

AMD / HIPNVIDIA / CUDAModels & quants

说明

## Summary

The WMMA scorer's launch order put all the key tiles of query-tile 0 first: each of the 16 query-tiles re-read the whole pooled array from DRAM because L2 (8 MiB) could not hold it between their turns. Swapping the grid's dimensions (x = the query tiles, y = the key tiles) schedules the 16 CTAs that share a key tile together, so L2 holds each tile for all of them and DRAM reads the keys once.

**The output is bitwise identical**: only the scheduling order of independent CTAs moves.

Opt-in `STRATA_SELECT_GRID=2` (adaptive: swaps launches deeper than `STRATA_SELECT_GRID_MIN` blocks, default 8192 = a 32K context), =1 forces the swap at any size, 0 or unset keeps 0.1.45's order.

## Measurements (gfx1151, Radeon 8060S, Strix Halo)

**Kernel level** (`qsa_select_bench`, 256 queries, capacity 524288):

| blocks | old order | swapped | delta |
|---|---:|---:|---:|
| 98K | 7.0 ms | 5.1 ms | **-27%** |
| 8K | 1.9 ms | 2.0 ms | +5% (L2-marginal — hence the adaptive bound) |

**End-to-end** (interleaved, all arms with `STRATA_GDN_CHUNKED=1`):

| depth | mainline | +grid-swap | delta |
|---|---:|---:|---:|
| 98K (x2 rounds) | 984.4 | 986.9 | +0.3% |
| 512K (x3 rounds) | ~800 | ~815 | **+1.8%** |

The 98K effect is small because many batches in a 98K prompt are below the 8192-block threshold; at 512K every batch is deep and the swap always engages.

## Scope

- One file changed: `src/kernels/cuda/qsa_select.cu` (+26/-4)
- `STRATA_SELECT_GRID=0|1|2` opt-in; default 0 = 0.1.45's order, bitwise identical
- `STRATA_SELECT_GRID_MIN` (blocks) tunes the adaptive threshold
- Independent of #1749's wide top-k and of SPLIT2 (they optimize different stages)

本站相关内容

相关页面的快捷入口。