Pull requests / #1100
#1100 s2_qpn8: opt-in QPN8 expert GEMV on Volta m8n8k4 (M >= 4)
open · @ATIVX928 · 0 comentários · No GitHub
BenchmarksSetup & installMulti-GPUAMD / HIPNVIDIA / CUDAModels & quants
Descrição
> Resubmitted from #1024: the original was auto-closed by the 2026-10-06 history cleanup. Rebased onto the new `main` (82f46a8); build and parity re-verified on 2x V100. ## What QPN8: a load-time repack of the Q2_0 expert weights at admission plus an m8n8k4 expert GEMV for Volta, split out of #627. The repack doubles the slot at admission and the GEMV builds the per-window A fragment and runs the mma chains; the DP4A path remains the fallback. The entry point pays off only when a window has M >= 4 expert entries; at M < 4 it is slower, so it stays **opt-in: `STRATA_QPN8=1`, default off**. ## Gating The m8n8k4 code is compiled under `#if !defined(__HIPCC__) && defined(STRATA_EXPERIMENTAL_SM60)` with stubs that fall back to DP4A elsewhere. The ready-made engine is unchanged. ## Measured V100-SXM2-16GB, CUDA 12.8, `s2_qpn8_parity --bench`, per window: | entries | DP4A | QPN8 | ratio | | --- | ---: | ---: | ---: | | 10x8 | 109.3 us | 90.6 us | 1.21x | | 10x4 | 73.5 us | 83.7 us | 0.88x | | 10x2 | 56.2 us | 77.0 us | 0.73x | | 10x1 | 50.1 us | 75.6 us | 0.66x | ## Tests - `s2_qpn8_parity` PASS with `STRATA_QPN8=1`: the quantized intermediates are byte-identical (0 of 30600 bytes differ), the FP32-level outputs are within their bound (worst abs 9.5e-6). - `s2_expert_grouped_parity` still bitwise PASS. - Default (QPN8 unset) skips/PASS. ## 32K end-to-end Same setup with `STRATA_QPN8=1`: prefill 1544.4 single / 2197.0 dual vs main's 1557.1 / 2207.0, and decode within its +-15% session noise. The MTP verify window is around M=5, between the measured 10x4 (0.88x) and 10x8 (1.21x) points. The entry point stays opt-in. ## Known gap The per-stage sizing loop for a layer split (~`generate.cpp:3514`) still prices `lay.max_blob`, as in the original #627 commit, so a Volta layer split with `STRATA_QPN8=1` can over-pick a stage and fail `open`. The rest of the admission/verify/generate accounting is updated.
No site
Links install, modelos, releases.