贡献 / #2080

#2080 qsa_select: the FP32 tiled block scores below sm_80 (PR #1091) on by default; STRATA_SELECT_SIMT=0 keeps the warp kernel

open · @imanu86 · 0 评论 · 去 GitHub 看

AMD / HIPNVIDIA / CUDAModels & quantsWindows

说明

## Title
Issue: none (a default change below sm_80, with the numbers).

## Summary
#1091 (konijiwa110) added the FP32 tiled QSA block-scores kernel for the prompt path below sm_80 and it was merged
**opt-in** (`STRATA_SELECT_SIMT=1`). This turns it **on by default** on compute capability < 8.0; `STRATA_SELECT_SIMT=0`
keeps the warp kernel. sm_80 and newer keep their TF32 tensor-core kernel; HIP is untouched.

## Why
On Turing the warp kernel runs at ~1 TFLOPS against ~6.9 for the tiled one, and the QSA block scores are a visible
share of a long prompt. #1091 measured +22% at 128K and +44% at 248K prompt speed on an RTX 2080 Ti; here, with the
two sm_75 changes of the same series, a 131K prompt on the 2080 Ti stage went from 100 s to 75 s. The ready-made
engine is what most Turing users run, and they do not read the environment table.

## What changed
`src/kernels/cuda/qsa_select.cu` (`qsa_block_scores_tc`: the `simt` default), `include/strata/kernels/qsa_select.hpp`
(the comment), `src/prefill/prefill.cpp` (the log line says which kernel ran).

## Bit for bit
No: the tiled kernel sums in another order than the warp kernel, as the opt-in already did. The prompt's selection
can differ at the rounding level.

## Measurements
RTX 2080 Ti 22 GB + RTX 3060 12 GB, Windows 11, CUDA 12.6, layer split (the 2080 Ti holds layers 16-47), Qwen3.8
IQ3_XXS: 131K prompt 100 s -> 75 s together with the 16-byte dequant and hc16 (separate PRs). TITAN RTX, #1981: the
community report ran with the switch on.

Built and started on v0.1.43 (RTX 2080 Ti + RTX 3060 layer split, Windows): the engine serves as before, same answer. The prompt-time figures come from this fork's daily build, where the three Turing prefill changes have run together since 0.1.40.

Part of a series from a public patch registry (`stack/` in the fork imanu86/Strata-2080Ti), see #2073.

本站相关内容

相关页面的快捷入口。