#2227 --mtp-hnorm stream beats the default on sm_89 (IQ3_XXS): draft acceptance 82%->86%, decode +3.8%/+2.3% at 6K/70K, neutral at 120K, answers byte-identical

Description

**Summary.** On an RTX 4090D (sm_89, Windows 11, 48 GB, PCIe Gen4 x4), Strata 0.1.43.1 with Qwen3.8-Flash-Next IQ3_XXS (native pack, 24,576/24,576 experts in VRAM) was measurably faster with `--mtp-hnorm stream` than with the default (`pooled`). The draft acceptance rate rises about 4 points and the decode follows; at 120K context the two are inside the noise band. Answers are unchanged (byte-identical on our greedy set).

**Rig / flags (nothing else differs between arms).**
- RTX 4090D 48 GB, Windows 11 (WDDM), Gen4 **x4**, 128 GB RAM, NVMe; engine 0.1.43.1 release build (`strata-windows-x64.zip`, archs 75/86/89/120, CUDA 13.0).
- Qwen3.8-Flash-Next GSQ-RCO IQ3_XXS (`--pack` + `--native` + `--ple-gguf`), `--ple-io direct`, `--embd-gguf` BF16 embeddings.
- Serving: `--spec 4 --spec-min-p 0.70 --mtp <rt>` (shipped draft vocab, 106,299 ids) `--prefill auto:32768 --pcie-frac 0.35 --kv int8 --kv-resident 32768 --prompt-cache 12 --conversation-cache-mib 32768 --conversation-cache-slots 8 --vram-reserve-mib 700`.
- Each arm is a config + engine restart; arms were interleaved (default → stream → default → stream) in one session because this box drifts ±4-6% across boots.

**Method.** One fixed filler prompt read to 6K / 70K tokens, then 5 greedy 384-token generations per arm (`bench/strata-decode-probe.py` style); the engine's own per-request line supplies `generated in X ms (Y tok/s), drafts accepted A of B` — the table shows the median of the 5 reps. 120K: the repo's own 3-needle prompt, 2 requests per pass, greedy.

**Results.**

| context | default (pooled) | `--mtp-hnorm stream` | delta |
|---|---:|---:|---:|
| 6K | 219.9 / 219.4 tok/s, accept 82% / 82% | **228.0 / 228.2**, accept **86% / 86%** | **+3.7% / +4.0%** |
| 70K | 203.0 / 203.3, accept 81% / 81% | **207.6 / 208.0**, accept **85% / 85%** | **+2.3% / +2.3%** |
| 120K needles | mean 220.2 (214.7, 225.6), accept ~89.5% | mean 220.2 (220.2, 226.6, 211.7, 222.4), accept ~88.5% | neutral |

The default arm is very reproducible across the interleave (219.9/219.4 and 203.0/203.3), so the 6K/70K gap is wider than its own band; at 120K the two overlap.

**Quality.** The draft only proposes and every token is still verified against the main model, so nothing should change — and nothing did on our checks: six greedy answers (Cantonese/English short, code, Cantonese/English long, translation) byte-identical between arms and against 0.1.43; 32K needle 3×3 = 9/9 and 120K needle 2×3 = 6/6 on both arms; a Cantonese temp-0.3 sanity prompt non-empty 6/6.

**Why (a guess).** `stream` is llama.cpp's `qwen4exp` MTP graph (one RMS per hyper-connection stream); the default pools the four into one (vLLM's `qwen4_exp`). This model's MTP head comes from the llama.cpp-side family, so the per-stream norm may match how the draft layer was trained.

**Two extra data points (may save someone a run).**
- `STRATA_MTP_FULL_HEAD=1` (draft over the full 248,320-token vocabulary): acceptance 86% → 90% at 6K, but decode falls 228.0 → 223.2 (−2.1%). The bigger head costs more than the extra acceptances earn, so the 106,299 subset stays.
- `STRATA_MTP_TOP2=1`: at rejected drafts the runner-up was the token the model picked **0 of 14** times (`rejected/runner-up by depth 6/0 4/0 4/0 0/0` over ~137 windows) — the remaining rejections are not near-misses.
- For completeness: `--lookup-chain 3` measured ±0.05% here (219.8/202.9 vs 219.9/203.0); its counters show 1-4 of ~148 windows carrying a chain and 0-3 chained drafts accepted per request.

**Suggestion.** If other sm_89 (or NVIDIA) cards confirm `stream` ≥ `pooled`, it may be worth a line in the MTP docs — or even flipping the default for NVIDIA. Happy to re-run this on request with different flags or a specific protocol.