#1889 Verify window: the mixer's input-only work on graph branches (STRATA_SIDE_MIXER=1/2, opt-in)

说明

## Title
Issue: none (an opt-in speed change)

## Summary
`STRATA_SIDE_MIXER=1` or `=2` (opt-in, off by default) records the mixer's work that reads only the layer's input on
branches of the verify window's CUDA graph, beside the projections that the recurrence and the attention wait for:

- `=1`: the GDN gates (a/b) on one branch, joined before the recurrence; the QSA indexer's chain (key projection, tail
  snapshot, key append, query projection with norm and RoPE, block scores, top-k) on it, joined before the attention.
- `=2`: also the GDN gate projection (z) and the QSA queries' chain (projection, q/gate split, norm, RoPE, rotation) on a
  second branch. A branch that projects reads F4's interleaved q8_1 copy, written on the main stream before the fork, and
  qkv is recorded before the branch so that its grid is dispatched first.

Each step is the in-line code with its stream as a parameter, so the outputs are bitwise the same (exact7 with the switch
on, below). `STRATA_DF_BRANCH` is the same idea for fewer of these steps; it is off whenever this switch is set. The idea
is from Eddoursul's fork (34a35db, b063c13).

**Measured** 2026-10-10 on one PC: RTX 5080 16 GB + RTX 4060 Ti 8 GB (driver 610.57.04, CUDA 13.3), Ryzen 9 9950X3D (16 cores), 96 GB DDR5, Linux 7.0; Qwen3.8-Flash-Next GSQ-RCO IQ3_S at a 131,072-token context. Served, stock against stock + this patch only, A B B A with a fresh server per arm (the model files evicted from the page cache before each), both arms with `--expert-cache auto --prefill auto:16384 --spec 4 --spec-min-p 0.5 --mtp --max-context 131072 --kv int8 --kv-resident 32768 --pcie-frac 0 --ple-inflight 128 --vram-reserve-mib 713` with `STRATA_IO_THREADS=64 CUDA_MODULE_LOADING=LAZY STRATA_GR_V3=1 STRATA_PF_FUSED=1 STRATA_MMVQ_IL=1 STRATA_ADAPT_LAG=2 STRATA_DISJOINT_ADAPT=1` (the stock flags of my community benchmark report, #1681); the patch's arm adds `STRATA_SIDE_MIXER=2`. Requests carry no sampling fields, so the server's configured defaults apply: temperature 1.0, top-p 0.95, top-k 20, min-p 0.05, reasoning effort high. Decode screen: long reasoning math and code (4,096 tokens each), five short questions, a 32K prompt with 768 tokens out; decode figures are the engine's own (accepted output tokens / decode time). Prompt-read screen: time to first token for prompts of 1K x3, 4K x3, 32K x2 and 120K x2 tokens (each a different text, 768 tokens out; its decode column comes from short sampled answers and varies more than the decode screen).

In short (this patch against the same stock build, means of the two A B B A arms each):

| setup | decode | ms per verify round |
|---|---|---|
| RTX 5080 alone, v0.1.42, upstream's defaults (tail skip on) | +1.3 % | -4.4 % |
| RTX 5080 alone, v0.1.42, `STRATA_ROUTE_TAIL_SKIP=0` | +4.2 % | -2.9 % |
| RTX 5080 + RTX 4060 Ti helper, v0.1.41 (before the rebase) | +1.7 % | -2.6 % |

*The RTX 5080 alone, against v0.1.42 `61b3fb5d` with upstream's defaults: v0.1.42 turns on `STRATA_ROUTE_TAIL_SKIP=7` for single-GPU serve, so both arms skip low-ranked missed experts (what single-card users get).*

| metric | v0.1.42: arm 1 / arm 4 -> mean | v0.1.42 + this patch: arm 2 / arm 3 -> mean | patch vs v0.1.42 |
|---|---|---|---|
| decode (tok/s) | 131.2 / 134.4 -> **132.8** | 136.7 / 132.3 -> **134.5** | +1.3 % |
| ms per verify round | 15.06 / 15.31 -> **15.18** | 14.70 / 14.33 -> **14.52** | -4.4 % |
| tokens per round | 1.976 / 2.057 -> **2.016** | 2.010 / 1.896 -> **1.953** | -3.2 % |
| acceptance | 0.642 / 0.666 -> **0.654** | 0.641 / 0.639 -> **0.640** | -2.2 % |
| long reasoning math (tok/s) | 149.5 / 152.1 -> **150.8** | 154.8 / 156.7 -> **155.7** | +3.3 % |
| long reasoning code (tok/s) | 120.8 / 125.2 -> **123.0** | 126.0 / 116.1 -> **121.1** | -1.6 % |
| TTFT 32K (s) | 7.72 / 7.73 -> **7.73** | 7.72 / 7.72 -> **7.72** | -0.1 % |
| all requests, total time (s) | 88.1 / 92.0 -> **90.0** | 86.1 / 85.2 -> **85.7** | -4.8 % |

| prompt size | v0.1.42: TTFT s / decode tok/s | v0.1.42 + this patch: TTFT s / decode tok/s | TTFT, patch vs v0.1.42 | decode, patch vs v0.1.42 |
|---|---|---|---|---|
| 1K (6 runs per side) | 0.81 / 122.2 | 0.82 / 124.3 | +0.5 % | +1.8 % |
| 4K (6 runs per side) | 1.23 / 120.8 | 1.23 / 126.8 | -0.3 % | +5.0 % |
| 32K (4 runs per side) | 7.66 / 122.0 | 7.66 / 124.0 | -0.0 % | +1.7 % |
| 120K (4 runs per side) | 29.65 / 144.1 | 29.64 / 140.1 | -0.0 % | -2.8 % |

*The RTX 5080 alone, against v0.1.42 with `STRATA_ROUTE_TAIL_SKIP=0` in both arms (the tail skip off: the answers of a build without it).*

| metric | v0.1.42: arm 1 / arm 4 -> mean | v0.1.42 + this patch: arm 2 / arm 3 -> mean | patch vs v0.1.42 |
|---|---|---|---|
| decode (tok/s) | 113.4 / 113.5 -> **113.5** | 118.3 / 118.1 -> **118.2** | +4.2 % |
| ms per verify round | 17.24 / 17.75 -> **17.50** | 17.01 / 16.98 -> **16.99** | -2.9 % |
| tokens per round | 1.955 / 2.015 -> **1.985** | 2.012 / 2.006 -> **2.009** | +1.2 % |
| acceptance | 0.634 / 0.653 -> **0.644** | 0.668 / 0.656 -> **0.662** | +2.9 % |
| long reasoning math (tok/s) | 131.6 / 127.3 -> **129.4** | 135.4 / 136.4 -> **135.9** | +5.0 % |
| long reasoning code (tok/s) | 105.2 / 108.9 -> **107.0** | 110.3 / 108.8 -> **109.6** | +2.4 % |
| TTFT 32K (s) | 7.73 / 7.73 -> **7.73** | 7.72 / 7.73 -> **7.72** | -0.0 % |
| all requests, total time (s) | 101.2 / 100.5 -> **100.8** | 96.7 / 97.9 -> **97.3** | -3.5 % |

| prompt size | v0.1.42: TTFT s / decode tok/s | v0.1.42 + this patch: TTFT s / decode tok/s | TTFT, patch vs v0.1.42 | decode, patch vs v0.1.42 |
|---|---|---|---|---|
| 1K (6 runs per side) | 0.84 / 104.3 | 0.82 / 112.6 | -2.6 % | +7.9 % |
| 4K (6 runs per side) | 1.25 / 104.4 | 1.25 / 107.9 | +0.0 % | +3.4 % |
| 32K (4 runs per side) | 7.67 / 104.1 | 7.66 / 104.3 | -0.0 % | +0.2 % |
| 120K (4 runs per side) | 29.65 / 117.4 | 29.65 / 118.9 | +0.0 % | +1.3 % |

*The RTX 5080 with the RTX 4060 Ti as the helper expert cache (`--expert-cache-device1 3300`), measured on v0.1.41 `fb58e0db` with this change on v0.1.41, before the rebase. The tail skip does not apply with a helper GPU.*

| metric | v0.1.41: arm 1 / arm 4 -> mean | v0.1.41 + this patch: arm 2 / arm 3 -> mean | patch vs v0.1.41 |
|---|---|---|---|
| decode (tok/s) | 125.5 / 124.6 -> **125.1** | 127.4 / 127.0 -> **127.2** | +1.7 % |
| ms per verify round | 15.98 / 16.15 -> **16.07** | 15.71 / 15.58 -> **15.65** | -2.6 % |
| tokens per round | 2.005 / 2.013 -> **2.009** | 2.002 / 1.979 -> **1.990** | -0.9 % |
| acceptance | 0.647 / 0.662 -> **0.655** | 0.650 / 0.644 -> **0.647** | -1.2 % |
| long reasoning math (tok/s) | 141.0 / 140.8 -> **140.9** | 144.2 / 141.5 -> **142.9** | +1.4 % |
| long reasoning code (tok/s) | 116.7 / 114.1 -> **115.4** | 117.9 / 118.6 -> **118.3** | +2.4 % |
| TTFT 32K (s) | 7.73 / 7.73 -> **7.73** | 7.72 / 7.72 -> **7.72** | -0.1 % |
| all requests, total time (s) | 94.9 / 92.8 -> **93.8** | 92.8 / 89.1 -> **91.0** | -3.1 % |

| prompt size | v0.1.41: TTFT s / decode tok/s | v0.1.41 + this patch: TTFT s / decode tok/s | TTFT, patch vs v0.1.41 | decode, patch vs v0.1.41 |
|---|---|---|---|---|
| 1K (6 runs per side) | 0.80 / 131.4 | 0.79 / 126.6 | -1.0 % | -3.6 % |
| 4K (6 runs per side) | 1.23 / 122.3 | 1.23 / 120.7 | -0.0 % | -1.3 % |
| 32K (4 runs per side) | 7.67 / 119.2 | 7.66 / 122.3 | -0.1 % | +2.6 % |
| 120K (4 runs per side) | 29.69 / 127.2 | 29.68 / 144.2 | -0.0 % | +13.3 % |

*Against `STRATA_DF_BRANCH`* (on v0.1.41 `fb58e0db` before the rebase, the same day, the RTX 5080 alone, the same flags and decode screen; one run of six arms
in mirrored order, two servers per arm): stock v0.1.41 113.8 tok/s and 17.44 ms per verify round; `STRATA_DF_BRANCH=1` 115.2
(+1.2 %) and 17.21 ms; `STRATA_SIDE_MIXER=2` 118.5 (+4.1 %) and 16.89 ms, which is 1.9 % less per round than
`STRATA_DF_BRANCH`.

## What changed
- `src/core/verify.cpp`: `g_side_mixer()` reads the switch. `Verifier::init` creates the two branch streams and their
  four events only when it is set. In `record_window` the GDN gates and gate projection, and the QSA indexer and query
  steps, are written as lambdas that take their stream (`idx_key`, `idx_snap`, `idx_append`, `idx_query`, `idx_select`,
  `q_chain`). With the switch unset they run on the window's stream in the same order as before; with it set they run on
  the branches. `STRATA_DF_BRANCH`'s branches are not used while it is set.
- `include/strata/core/verify.hpp`: the streams and events.

Left alone: batch windows (each row reads its own slot's indexer, so they stay in line), the draft layer, the prompt
path, `STRATA_DF_BRANCH` itself, and every path with the switch unset (no stream or event is created, the same launches in
the same order).

## Extra Notes
**How the default path was checked.** exact7 (the CLI, seven fixed computations: `--adapt-swaps 0 --suffix-draft 0 --expert-cache 3500`) on v0.1.42 `61b3fb5d` and on this branch, both cards, the measurements' flags (above) plus `STRATA_PREFILL_CPU_SHARE=0` on both (with the CPU prompt-read share on, short-prompt outputs depend on timing and the stock build does not repeat itself; with it off, stock v0.1.42 run twice gave the same seven hashes; the CLI does not use v0.1.42's tail skip, whose default is set only at serve start-up). With the switch unset: **7/7 identical to v0.1.42**. With `STRATA_SIDE_MIXER=2`: 7/7 identical.

| exact7 item | v0.1.42 `61b3fb5d` | this branch, switch unset | switch on |
|---|---|---|---|
| math, greedy, 1,024 tokens | `02deb3f1` | `02deb3f1` (=) | `02deb3f1` (=) |
| math, sampled (seed 11), 1,024 tokens | `6c9b2d10` | `6c9b2d10` (=) | `6c9b2d10` (=) |
| code, greedy, 1,024 tokens | `ae3b089b` | `ae3b089b` (=) | `ae3b089b` (=) |
| code, sampled (seed 11), 1,024 tokens | `38e671a2` | `38e671a2` (=) | `38e671a2` (=) |
| 2K-token prompt + 64 tokens | `3e219376` | `3e219376` (=) | `3e219376` (=) |
| 32K-token prompt + 64 tokens | `651f56c8` | `651f56c8` (=) | `651f56c8` (=) |
| 112K-token prompt + 64 tokens | `a393ca0a` | `a393ca0a` (=) | `a393ca0a` (=) |

**The patch's own checks.**

- exact7 with `STRATA_SIDE_MIXER=2` (table above): 7/7 identical to v0.1.42. Every step on a branch is the in-line kernel on
  another stream, so the outputs are bitwise the same.

**Upstream's tests** (ctest, the whole suite, among them verify_parity, verify_batch_parity, spec_verify_parity, qsa_parity and spec_prob_test): v0.1.42 109/112 pass, this branch 109/112 pass; the same 3 fail on both (expert_parity, ple_parity, pool_test): they read model files from fixed paths that this PC does not have.

#905: `STRATA_DF_BRANCH` is off by default on Linux because a stage's window with branches stopped inside its graph when
an NVML query came between requests (two RTX 3090s, two RTX 2080 Tis). This change uses the same fork and join inside the
captured graph, so the same risk may apply to layer-split stages. I have not seen it here: `STRATA_SIDE_MIXER=2` has been
in my daily build since 2026-10-03 (Linux, open kernel modules, one stage on the RTX 5080 with the RTX 4060 Ti as the
helper expert cache), read by nvidia-smi every 15 minutes, and every second during 2-hour soaks.

Not tested:

- **HIP and SYCL builds.** `verify.cpp` is compiled by the HIP builds, but this PC has no ROCm (installing it would be a
  system change), so it is a draft. SYCL is its own tree and is not touched.
- Layer-split and multi-stage setups (#905's case), several requests at once (batch windows stay in line), and Windows.
- `STRATA_SIDE_MIXER=1` alone was not measured; the numbers are for `=2`.

Measured and drafted with Claude Code.