#5432 [Engine] Do not force WindowInvariant CPU experts on DFlash2 windows when MTP fallback is loaded

open · @bsorensen110 · 0 评论 · 去 GitHub 看

Server & APIMulti-GPUAMD / HIPNVIDIA / CUDAWindowsLinux

说明

## Problem and result

When DFlash2 runs with the MTP layer loaded beside it as the fallback (`--draft auto` / `use_mtp == true` with `req_df2 == true` in `serve` or `spec_uses_df2 == true` in `generate`) and `STRATA_SPEC_PROB=1` is enabled (including on `gfx12` where `arch_defaults.cpp` sets `STRATA_SPEC_PROB=1`), `mtp_q_applies` (`include/strata/spec/window_rule.hpp:35`) correctly omits the MTP proposal distributions `q` on DFlash2 verify windows (`!df2_active`). However, the immediately following `decode_pool_fn` selection (`src/program/generate.cpp:18858` in the `serve` loop and `src/program/generate.cpp:20650` in the CLI loop) and `gated_proposal` (`src/program/generate.cpp:17550`) evaluated `use_mtp && mtp.prob()` without checking `!req_df2` / `!spec_uses_df2`.

As a result, every DFlash2 verify window with an MTP fallback loaded forced CPU expert misses onto `invariant_win_pool_fn` / `drive_pool_multi_invariant` (`NativeArithmetic::WindowInvariant`, which sets `singleton_iq = true` and runs 1-token reference kernels per row instead of the multi-token grouped AVX2/AVX-512 IQ kernels) even though DFlash2 uses deterministic point-mass candidates (`mtp_q_applies == false`).

This change adds `strata::spec::mtp_window_invariant_applies(bool use_mtp, bool df2_active, bool mtp_prob)` in `include/strata/spec/window_rule.hpp` and gates `gated_proposal` and both `decode_pool_fn` sites on it, keeping multi-token grouped CPU expert kernels active whenever DFlash2 is the active drafter.

## Related discussion

None.

## Approach

- **`include/strata/spec/window_rule.hpp:39-42`:** Add `mtp_window_invariant_applies(bool use_mtp, bool df2_active, bool mtp_prob)` returning `use_mtp && !df2_active && mtp_prob`, alongside `mtp_q_applies`.
- **`src/program/generate.cpp:17550, 18858, 20650`:** Gate `gated_proposal` and `decode_pool_fn` (both `serve` and `generate` loops) through `strata::spec::mtp_window_invariant_applies(...)` using the request/session DFlash2 flag (`req_df2` in `serve`, `spec_uses_df2` in `generate`).
- **`src/spec/window_rule_test.cpp:105-108`:** Add unit test assertions covering all four `(use_mtp, df2_active, mtp_prob)` states of `mtp_window_invariant_applies`.
- **No behavior change for:** MTP-only probabilistic decoding (`use_mtp && !df2_active && mtp.prob()`), greedy decoding, or DFlash2-only runs without MTP loaded (`--draft-fallback none`).

## Validation

Tested on revision `e6609ae52` (based on `main` `54828c7ef`, Strata 0.2.10 / engine 0.1.57) on Linux (`7.3.1-1-cachyos-server`, AMD Radeon AI PRO R9700 `gfx1201` 32 GB VRAM, 128 GB RAM).

**Passed:**
- `g++ -std=c++20 -O2 -Iinclude src/spec/window_rule_test.cpp -o window_rule_test && ./window_rule_test` on `e6609ae52`: `window_rule_test: ok` (all 22 checks pass, including the 4 new `mtp_window_invariant_applies` cases).
- Base vs. fix probe on `54828c7ef`: for `(use_mtp=true, df2_active=true, mtp_prob=true)`, the prior inline condition `use_mtp && mtp_prob` returned `true` (selecting `invariant_win_pool_fn` while `mtp_q_applies(true, true, true, false, false, 4)` returned `false`); `mtp_window_invariant_applies(true, true, true)` returns `false`.
- `cmake --build build-hip --target window_rule_test spec_prob_test spec_admission_test strata -j16` and CTest `window_rule_test`, `spec_prob_test`, `spec_admission_test`: 3/3 pass (`0` failures).

**Failed:**
- none observed

**Not run:**
- CUDA and Metal builds (no NVIDIA or Apple Silicon hardware on this Linux HIP host; `include/strata/spec/window_rule.hpp` and `src/program/generate.cpp` are shared across CUDA and HIP).

## Limits

- Does not change DFlash2 candidate verification itself (which remains exact-match / sampled residual verification without MTP `q` lists, as intended by `mtp_q_applies`).
- When MTP itself is the active drafter (`df2_active == false`) and `STRATA_SPEC_PROB=1` is active on a sampled request (`temperature >= 0.05`), `mtp_window_invariant_applies` still returns `true` so prefix-invariant CPU expert arithmetic is preserved for MTP rejection sampling.

## Reproduction

```sh
g++ -std=c++20 -O2 -Iinclude src/spec/window_rule_test.cpp -o window_rule_test
./window_rule_test
```

本站相关内容

相关页面的快捷入口。