#6577 [Engine] apply NVFP4 per-expert down scale in grouped subwarp reduction

open · @li-yifei · 0 comentários · No GitHub

BenchmarksServer & APIAMD / HIPNVIDIA / CUDAModels & quantsWindowsLinux

Descrição

## Problem and result

In `engine/src/kernels/cuda/iq_kernels.cu`, `native_down_lds_kernel<TD>` handles grouped decode down projections for `STRATA_EXP_MODE=7` and `STRATA_EXP_MODE=8` (the default on `sm_120` Blackwell). For NVFP4 (`TD == 40`, `Fmt<40>::qk == 64`, `Fmt<40>::ipb == 2`) with `n_ff = 640`, `n_items = 20`, which selects the 8-lane subwarp path (`use_sub8`).

That `use_sub8` path wrote the reduced accumulator `v` directly to `out` without multiplying by the per-expert NVFP4 down-projection tail scale `sd = ((const float*) (blob + L.tail_off))[2]`, even though the 32-lane fallback in the same kernel and the mode 0 kernel both apply `sd`. Because NVFP4 `s_down` scales are typically small (`~1e-4` to `1e-2`), omitting `sd` inflated down-projection outputs by `1 / s_down` and corrupted generation on NVFP4 models whenever grouped mode 7 or 8 ran.

**Before:**
- On stock `v0.2.19` (`c6f6427d9`) on RTX 5090 with `qwen3.8-flash-next-huihui-nvfp4-gptq`:
  - With `STRATA_PREFILL_NVFP4=w4a8` and remaining SM120 defaults (including `STRATA_EXP_MODE=8`), a 31,992-token prompt returned garbled text (`draft_n_accepted: 0` of 15).
  - With `STRATA_QFUSE=0 STRATA_LFUSE=0 STRATA_DEQUANT_SIDE=0 STRATA_SIDE_MIXER=0` (leaving `STRATA_EXP_MODE=8` active), even a 19-token prompt returned garbled text (`draft_n_accepted: 0` of 4).
  - Switching off SM120 defaults (`STRATA_SM120_PERF=0 STRATA_PREFILL_NVFP4=w4a8`) restored exact answers on short, 32k, and 200k prompts.
- In `nvfp4_decode_sm120_test` linked against the unmodified `c6f6427d9` `iq_kernels.cu` object, both grouped mode 8 and grouped mode 7 failed immediately on `G=10 E/G=1` with `rel_rms=8060.792021 max_abs=10119.366077 FAIL` against mode 0.

**After:**
- `native_down_lds_kernel<40>` applies the per-expert `s_down` tail scale in the `use_sub8` path, matching mode 0 (`rel_rms=0.000000 max_abs=0.000000` across the tested grouped cases).

## Related discussion

Companion W4A4 prefill fix: #6576. Both changes were used in the combined end-to-end check below.

## Approach

- In `native_down_lds_kernel` (`engine/src/kernels/cuda/iq_kernels.cu`):
  - Read `e0 = grp_start[g]`, `e1 = grp_start[g + 1]` and return early when `e0 >= e1` (uniform across the block) before dereferencing `grp_ptr[g]` or `blob + L.tail_off`, keeping empty groups safe when `sd` is hoisted out of the entry loop.
  - Hoist `const float sd = (TD == 40 && L.tail_off) ? ((const float*) (blob + L.tail_off))[2] : 1.0f;` once per non-empty group so both the `use_sub8` and 32-lane paths share it.
  - Apply `if constexpr (TD == 40) v *= sd;` on `slane == 0` before storing in `use_sub8`, leaving non-NVFP4 instantiations (`TD == 20` IQ4_NL and `TD == 42` Q2_0) untouched on that path.
- In `engine/tests/cuda/nvfp4_decode_sm120_test.cpp`:
  - Use small per-expert `tail[2] = (1 + g % 4) * 0.00005f` scales so missing or misaligned per-group `s_down` scaling fails loudly.
  - Explicitly set `native_expert_set_mode(0, 0)` for the DP4A reference and poison output buffers (`0xff` / `0x7f`) before each launch.
  - Check grouped modes 8 and 7 against the mode 0 reference with a tighter grouped-specific tolerance (`1e-5`, while keeping `5e-3` for the `sm120 tensor core` path) and explicit finiteness / non-zero reference checks.
  - Add a 3-entry-per-group case (`run_case(24, 3, bench)`) alongside `E/G = 1, 2, 4, 8` so all four `switch (cn)` arms (`1, 2, 3, default/4`) in `native_down_lds_kernel` are exercised.

**Compatibility effects:**
- Changes no default switches or dispatch rules.
- For `TD == 20` (IQ4_NL) and `TD == 42` (Q2_0), `if constexpr (TD == 40)` compiles out the `use_sub8` multiply and `sd` remains constant `1.0f` in the 32-lane branch; empty groups now return before reading `grp_ptr[g]`. Non-NVFP4 packs were not run locally, so byte-identity on IQ4_NL / Q2_0 is inferred from code inspection rather than measured.

## Validation

**Environment:**
- Upstream base: `v0.2.19` (`c6f6427d9de9016fd92d13e29740f65625261df7`, engine `0.1.65`)
- Hardware: NVIDIA GeForce RTX 5090 (`sm_120`, compute capability 12.0, 32 GiB VRAM)
- OS / toolchain: Linux x86_64 (`7.1.5-76070105-generic`), GCC 13.3.0, NVIDIA driver `615.71.09`, CUDA `13.1` (`nvcc` V13.1.115), Release build (`-DCMAKE_BUILD_TYPE=Release -DCMAKE_CUDA_ARCHITECTURES=120-real`)
- Model pack (end-to-end checks): `qwen3.8-flash-next-huihui-nvfp4-gptq` (NVFP4 experts, `n_embd = 2560`, `n_ff = 640`, 48 layers, `--prefill auto:32768 --max-context 262144 --kv int8 --spec 6 --spec-min-p 0.7`)

**Passed:**
- *Negative controls (reproduction of original bug):* `nvfp4_decode_sm120_test` linked against the unmodified `c6f6427d9` `iq_kernels.cu` object failed as expected on both grouped modes:
  ```text
    NVFP4 sm120 tensor core G=10 E/G= 1: rel_rms=0.000000 max_abs=0.000000 OK
    NVFP4 grouped mode 8 G=10 E/G= 1: rel_rms=8060.792021 max_abs=10119.366077 FAIL
  ```
  and with mode 7 tested first:
  ```text
    NVFP4 sm120 tensor core G=10 E/G= 1: rel_rms=0.000000 max_abs=0.000000 OK
    NVFP4 grouped mode 7 G=10 E/G= 1: rel_rms=8060.792021 max_abs=10119.366077 FAIL
  ```
- *Regression test at PR head `0314f3ea0`:* `./nvfp4_decode_sm120_test` on RTX 5090:
  ```text
  NVFP4 sm120 tensor core G=10 E/G= 1: rel_rms=0.000000 max_abs=0.000000 OK
    NVFP4 grouped mode 8 G=10 E/G= 1: rel_rms=0.000000 max_abs=0.000000 OK
    NVFP4 grouped mode 7 G=10 E/G= 1: rel_rms=0.000000 max_abs=0.000000 OK
    NVFP4 sm120 tensor core G=24 E/G= 2: rel_rms=0.000000 max_abs=0.000000 OK
    NVFP4 grouped mode 8 G=24 E/G= 2: rel_rms=0.000000 max_abs=0.000000 OK
    NVFP4 grouped mode 7 G=24 E/G= 2: rel_rms=0.000000 max_abs=0.000000 OK
    NVFP4 sm120 tensor core G=24 E/G= 3: rel_rms=0.000000 max_abs=0.000000 OK
    NVFP4 grouped mode 8 G=24 E/G= 3: rel_rms=0.000000 max_abs=0.000000 OK
    NVFP4 grouped mode 7 G=24 E/G= 3: rel_rms=0.000000 max_abs=0.000000 OK
    NVFP4 sm120 tensor core G=32 E/G= 4: rel_rms=0.000000 max_abs=0.000000 OK
    NVFP4 grouped mode 8 G=32 E/G= 4: rel_rms=0.000000 max_abs=0.000000 OK
    NVFP4 grouped mode 7 G=32 E/G= 4: rel_rms=0.000000 max_abs=0.000000 OK
    NVFP4 sm120 tensor core G=32 E/G= 8: rel_rms=0.000024 max_abs=0.000629 OK
    NVFP4 grouped mode 8 G=32 E/G= 8: rel_rms=0.000000 max_abs=0.000000 OK
    NVFP4 grouped mode 7 G=32 E/G= 8: rel_rms=0.000000 max_abs=0.000000 OK
  nvfp4_decode_sm120_test: ok
  ```
- *Combined end-to-end check (pre-polish revision, with both this fix and the companion NVFP4 prefill quantization-grid fix active under all 13 SM120 defaults on RTX 5090):*
  - Short prompt (`"42"`), 31,992-token and 199,977-token retrievals (`"CEDAR7429"`), cached follow-ups (`"7429"`), 259,880-token near-limit retrieval (`"KITE8326 MORI5194 ELM2738"`), `/v1/responses`, code generation, automatic tool calling (`echo`), and synthetic 3-turn incident handoffs at ~32k and ~200k context (6/6 JSON checks) all passed.

- The complete changed translation unit at this head compiled for HIP `gfx1151` with ROCm 7.2 headers and LLVM/lld 21.

**Failed:**
- none observed on the patched build (stock `v0.2.19` negative controls failed as expected above).

**Not run:**
- Full HIP engine link and AMD GPU runtime tests.
- SYCL and Metal builds (SYCL uses its separate `engine/sycl/src/kernels/cuda/iq_kernels.dp.cpp`, which does not have `use_sub8` or `TD == 40` in `native_down_lds_kernel`; Metal does not compile `iq_kernels.cu`).
- Windows CUDA, non-`sm_120` CUDA architectures, non-NVFP4 `native_down_lds_kernel` formats (`IQ4_NL`, `Q2_0`), full `STRATA_BUILD_TESTS=ON` CTest suite, and `tools/check_cuda_spills.py`.

## Limits

- `nvfp4_decode_sm120_test` skips with exit code `77` on non-Blackwell GPUs (`!nvfp4_decode_sm120_available()`), so the new grouped mode 7/8 assertions inside that binary only execute on Blackwell `sm_120a`/`sm_121a` hardware.

No site

Links install, modelos, releases.