Pull requests / #3151
#3151 qsa: the default block-scores kernel without #2234's per-query keys (a gfx1151 verify window stalled on v0.1.45); the batch rows get their own kernel
open · @gt50 · 0 commentaires · Sur GitHub
BenchmarksSetup & installAMD / HIPNVIDIA / CUDAWindows
Description
On v0.1.45 a Radeon 8060S (gfx1151, Windows 11, ROCm 10.2.0a20260930) stalls in its first verify window when gfx1151's default switches are on. The engine prints `verify: an earlier window never finished on the GPU (#267)` and "the GPU rang 0", the watchdog restarts it after 60 s, and no token comes out. It happens with a stock config and with mine. The installed 0.1.40.2 engine, with the same switches and config, does not stall. **Cause:** f629fb04 (#2234), its `qsa_select.cu` change. `block_scores_multi_kernel` is the kernel every decode window's QSA block scores run on. #2234 gave it a by-value `QsaScoreRows` argument (16 pointers and a count) and a per-query branch for the opt-in `STRATA_BATCH_ATTN_ROWS` path. Both apply with that switch off. **Fix:** `block_scores_multi_body` is a template on `OWN`. - The default kernel (`OWN` false) is the pre-#2234 kernel again, with no `QsaScoreRows` argument and no branch. - `qsa_block_scores_rows` launches a new `block_scores_multi_rows_kernel` (`OWN` true). - One file: `src/kernels/cuda/qsa_select.cu`, +33/−14. ### How it was found All on that Radeon 8060S, Qwen3.8-Flash-Next UD-IQ4_XS, `--spec 4 --mtp`, `--kv int8`. Each trial was a fresh start, then a 14-token request with `max_tokens` 8. No answer in 30 s counted as a stall; a normal answer takes 0.4-0.7 s. - **The switches:** 26 trials with `STRATA_GFX1151_DEFAULTS=0` and subsets turned on. `STRATA_PLE_BATCH` is on in every set that stalled. It needs `STRATA_LFUSE` and `STRATA_TSUM` or `STRATA_GDN_SPLIT` beside it, and other switches change which set is enough. With the other 18 defaults on, `STRATA_PLE_BATCH=0` stops the stall. - **Not every start stalls:** one build stalled in 6 of 7 starts (c8265fce, the commit just before v0.1.45 on main). So each step below used 3 starts. - **Commits:** a bisect from v0.1.40.2 (good) to v0.1.45 found the first bad main-line merge, e8b1edbc "perf-PR group batch". Its parent 5a6f4ed5 answered 3 of 3. Inside the merge, the commits from f629fb04 up to f755a06d do not build on HIP (`qsa_decode_attn.cu`; f755a06d fixes that). So each part was applied alone to the merge base 296b59d2: | On 296b59d2 | Starts | |---|---| | a2de7446 (PleTable lock) | 3 of 3 answer | | f629fb04 + 739bc948 + f755a06d | stall | | only the `qsa_decode_attn` files (f755a06d's refactor) | 3 of 3 answer | | only f629fb04's GDN change (`verify_kernels`) | 3 of 3 answer | | **only f629fb04's `qsa_select.cu` / `.hpp` change** | **stall** | The f-145-fix3 merge with #2622 answered 3 of 3, so the PLE gather change is not the cause. Several 0.1.45 merges do not build on Windows HIP without e6ad6c81's 4-line `hip_compat` fix, which these tests applied. ### Checked with this change On v0.1.45 (477ac13c) plus this commit, on the same machine: - **The stall trial:** 5 of 5 starts answer with all 19 gfx1151 defaults on. - **The parity tests:** `batch_rows_parity` passes (it checks `qsa_block_scores_rows` at 1, 2, 4 and 8 rows against one call per row, bitwise). So do `qsa_parity`, `qsa_topk_parity`, `hip_native_qsa_score` and `verify_batch_parity`. `qsa_scores_gfx1100_parity` builds and skips itself, since it is gfx1100 only. - **Answers:** three greedy answers (284-344 tokens) are byte-identical to v0.1.45 with `STRATA_PLE_BATCH=0`. - **Speed:** decode 34.9 / 48.9 / 40.3 tok/s, against 34.9 / 48.5 / 40.8 with the workaround. ### Not covered - **Other backends:** not built for CUDA or SYCL. The code uses `if constexpr`, as other `.cu` files here already do. - **The mechanism:** why the argument and branch hang the window only beside those switches is not explained here. The tests show this change caused it, and that taking it out of the default kernel stops it. - **Multi-row speed:** the per-row path's speed with `--batch` and `STRATA_BATCH_ATTN_ROWS=1` was not measured. Only its parity test ran. - **Unrelated test build:** a full `tools\hip\build_windows.bat tests` stops at `hc_persist_parity.cpp`. It uses POSIX `setenv`/`unsetenv`, which this change does not touch. Found while testing #1916 on v0.1.45. cc @yi-sudo (#2234) 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Sur le site
Liens install, modèles, releases.