#1842 0.1.41: 128K decode is 18% slower than 0.1.40.3 on an RTX 4090 (sm_89); STRATA_MMVQ_IL=0 gives all of it back

Description

**On a Windows RTX 4090 (sm_89, IQ3_S pack) the 128K-context decode drops from 90.3–95.4 tok/s on v0.1.40.3 to 73.2–78.4 on v0.1.41 — four runs, no overlap between the arms — while prefill at the same context is unchanged (3705.5–3774.7 vs 3745.0–3749.4). On the same v0.1.41 binary, `STRATA_MMVQ_IL=0` restores 90.7 and 91.9 tok/s, inside the v0.1.40.3 band, and short-context decode keeps its v0.1.41 gain (8K per-request decode 105.3–118.6 with IL off, 89.2–109.6 with IL on, 63.6–83.3 on v0.1.40.3). The engine's own table says why this card in particular: `native_mmvq.cu:2510` — "sm_89 (Ada) has none yet: nobody measured it (we have no such card), so it takes the shared table", and that shared table was read off an RTX 3060 (sm_86) and an RTX 5070 (sm_120) with the rule "the worse of the two cards still takes at least 3% off `native_mmvq`'s time". Limits: one 4090 rig, one quant, one model; the same A/B on an A100-class A30 (sm_80, UD-Q4_K_XL, Linux) shows **no effect at all**, so this is a table-tuning miss on an unmeasured card, not a general regression.**
## Setup
Windows 11, RTX 4090 24 GB (+ a GTX 1660 Ti holding only the vision encoder), 128 GB RAM, CUDA 12.9, engine from the release zip (`source: release`, `experimental: true`).
- v0.1.41 `strata.exe` md5 `39898e491e796c51b7d042783817d2ae`; v0.1.40.3 md5 `5ad3617a7a9a5c7fb330e6add75fc87c`.
- Pack: Qwen3.8-Flash-Next IQ3_S. Engine args (identical in both arms, verified from the startup block, which is byte-identical apart from the version string): `--spec 4 --spec-min-p 0.5 --mtp <rt> --max-context 262144 --kv int8 --kv-resident 32768 --expert-cache auto --prefill auto --ple-io ram --pcie-frac 0.0 --vram-reserve-mib 700 --vision` → `8251 slots / 15.65 GiB`, `424 MiB of VRAM free`, `verify: window up to 6 tokens`, `draft head over 106299 tokens`.
- Method: `tools/staged_bench.py` with **one salt label across all arms**, so the prompt bytes are identical; arms alternate order per round; each arm's binary is re-checked by md5 after the swap. Decode is the engine's own per-request line right after the 128K prefill (48 tokens, `--ask` = the prime list).
## The 128K decode, six samples
| arm | 48 tokens | decode tok/s | verify windows (steps) | ms per step |
|---|---|---|---|---|
| v0.1.41, IL on (default) r1 | 656 ms | 73.2 | 45 | 14.6 |
| v0.1.41, IL on (default) r2 | 612 ms | 78.4 | 42 | 14.6 |
| **v0.1.41 + `STRATA_MMVQ_IL=0`** direct | **529 ms** | **90.7** | 41 | 12.9 |
| **v0.1.41 + `STRATA_MMVQ_IL=0`** via a proxy | **522 ms** | **91.9** | 37 | 14.1 |
| v0.1.40.3 r1 / r2 | 503 / 532 ms | 95.4 / 90.3 | 37 / 41 | 13.6 / 13.0 |
The step count is **not** the effect: across all six samples it moves 37–45 on its own (48 tokens is a small sample), so read it aligned — at 41 steps IL-on gives 78.4 and IL-off gives 90.7 (+15.7%). The per-step cost is the effect, and that is also what kills the natural first guess: **this is not KV streaming or bandwidth.** The KV ledger is the same in every arm (`KV streaming: 88.77% of 382044 block reads hit VRAM, 172.8 MiB read from RAM` in the IL-off run; 88.4–89.3% in the IL-on runs), the expert hit rate is the same (85.5–90.1%), and prefill at 128K never moves (3786.0 / 3788.0 tok/s with IL off).
## The A30 (sm_80) is unaffected — same A/B, env-only arms on one binary
A100-class A30 24 GB, Xeon Gold 5320, Linux, CUDA 12.4 build, UD-Q4_K_XL pack, `0c085b0de2386b30e8c11cbb6a21974c`, 5184 slots, 2 rounds × 2 arms, order alternated, same salt:
| A30 | IL on (default) | `STRATA_MMVQ_IL=0` |
|---|---|---|
| 128K prefill tok/s | 1566.4 / 1554.3 | 1553.8 / 1552.9 |
| **128K decode tok/s** | **63.0 / 62.8** | **62.1 / 63.4** |
| 512 / 1k / 8K prefill | 438.6 / 561.7 / 1170.6 | 439.3 / 561.2 / 1168.5 |
| decode best (3 × 128 at 8K) | 79.4 | 79.3 |
So the same default-on path is free on sm_80 and costs ~18% on sm_89 at long context.
## Why I think it is the table, not the kernel
`native_mmvq_il_supported()` gates IL to types IQ4_XS / Q4_K / Q5_K / Q6_K, 2–4 columns, sm_80+ (or sm_70 with the opt-in), and `g_multi_exact`. The rows-a-warp choice comes from `native_mmvq_il_rows_for(cc, …)`, whose only per-architecture entry is `kIlRows_70`; **every other sm_80+ card, sm_89 and sm_80 included, uses `kIlRowsShared`**, measured on an RTX 3060 and an RTX 5070 with a "worse card still ≥3% faster" rule. A 4090 is neither card, and the cells that matter for a decode window are exactly the 2–4-column dense projections. The source comment even names the fix and the no-rebuild override: `mmvq_il_parity --bench --emit-table` → paste as `kIlRows_89`, or `STRATA_MMVQ_IL_ROWS="type:ncols:r0,r1,r2,r3,r4;…"`.
## Ruled out
- Draft acceptance / speculative depth: the acceptance column is equal or better on 0.1.41 in every context bucket over 3,240 production requests on this box; `--spec 4` identical; coupled drafting and `STRATA_SPEC_PROB` / `SPEC_GUMBEL` / `SPEC_MIN_TEMP` are off by default and not enabled here.
- Every other new long-context path in 0.1.41 is gated away on this card: PDL and the decode top-k thread-block cluster need sm_90+; the FP32 tiled block scorer needs <sm_80 and `STRATA_SELECT_SIMT=1`; the pre75 attention kernel needs <sm_75; `STRATA_PIN_GUARD` / `PIN_RESERVE_GIB` are inside `#if defined(__linux__)`; `file_cache_keeps`' hot fraction explicitly keeps the Windows rule; `STAGE_PIN` is `STAGE_PIN_DEFAULT 0`.
- KV residency and expert cache: identical args, identical hit rates, identical `VRAM free` line.
- The `3 × 128 at 8K` decode metric cannot see this at all: it reads 132.5–138.1 on 0.1.41 and 134.9–142.5 on 0.1.40.3, i.e. "no difference", while the 128K decode is 18% down. Worth saying about the release gate: it caught `STRATA_PIN_GUARD`/`STAGE_PIN` corrupting IQ3_S decode, but a default-on decode kernel that costs 18% only at long context passes it, because the gate's decode number is a short-context one.
## What I am asking
1. Is IL default-on intended for sm_80+ cards that have no measured table, or only for the shapes where the shared table was actually measured? On this 4090 the honest answer for long-context decode is that the shared table is a guess that loses ~18%.
2. Would a measured `kIlRows_89` (and an sm_80 entry, since the A30 also inherits the shared table) be welcome? We own a 4090 and an A30 and can run `mmvq_il_parity --bench --emit-table` if a Windows/CUDA 12.9 build recipe for that binary is documented (`docs/MMVQ_IL_TABLE.md` is referenced in the source comment).
3. If a table is not coming soon, is `STRATA_MMVQ_IL_ROWS` with the 4090's costly cells set to 0 the intended interim answer? We are running `STRATA_MMVQ_IL=0` in production on this box now, which is coarser than needed.
We can also sweep `STRATA_MMVQ_IL_ROWS` per type on this 4090 (four 128K decode runs, ~2 minutes each) and report which cells cost, if that helps narrow it without an Ada card in your lab.
Engine log lines for all six 128K runs are attached (`strata-iq3_s.log`, the `prompt 158896 tokens = …` lines with their `drafts accepted A of B` and `KV streaming:` follow-ups).