Pull requests / #2081

#2081 dequant: the prompt projections' BF16 dequant with 16-byte stores on sm_75 (bit-exact; STRATA_TC_SM75=0 keeps the per-group kernel)

open · @imanu86 · 0 コメント · GitHub で見る

NVIDIA / CUDAModels & quantsWindows

本文

## Title
Issue: none (bit-exact speed change on sm_75).

## Summary
On compute capability 7.5 the prompt path's BF16 projections are dequantized by a kernel that writes 16 bytes per
thread instead of 2 bytes every 64. A conversion with no arithmetic: the output is **bit for bit** the per-group
kernel's. `STRATA_TC_SM75=0` keeps the old kernel; other cards are untouched. 0.1.43's #1720 coalesced the Q8_0 path
the same way; this covers the BF16 projections on Turing.

## What changed
`src/kernels/cuda/dequant_bf16.cu` (the sm_75 kernel and its dispatch), `include/strata/core/tc_sm75.hpp` (the
`STRATA_TC_SM75` switch, shared with the hc16 PR).

## Measurements
Part of the 100 s -> 75 s of a 131K prompt on the RTX 2080 Ti stage (with SIMT by default and hc16). Values identical
(memcmp of the dequantized images in the fork's test build).

Built and started on v0.1.43 (RTX 2080 Ti + RTX 3060 layer split, Windows): the engine serves as before, same answer; the dequantized images were compared byte for byte in the fork's test build when the change was made.

Part of a series from a public patch registry (`stack/` in the fork imanu86/Strata-2080Ti), see #2073.

関連リンク

インストール・モデル・リリースへの站内リンク。