#2074 0.1.42 -> 0.1.43: decode -3.4% on RTX 4090 48 GB, IQ3_S, --resident-experts (MTP accept unchanged, prefill flat)

Descrição

**Summary.** Single RTX 4090 48 GB, native IQ3_S pack, `--resident-experts`: 0.1.43 decodes **-3.4%** vs 0.1.42 over 60 paired SPEED-Bench samples (2 runs vs 4 runs). MTP accept rate is unchanged, so this is per-token cost, not speculation. 0.1.43's advertised decode win (`ADAPT_OVERLAP`) is unreachable on this configuration, so the cost lands and none of the benefit does. Related direction: #2010, #2024.

## Setup

| | |
|---|---|
| GPU | RTX 4090 48 GB (modded), 450 W limit, `clocks.sm` 2700 / max 3105, `clocks_event_reasons.active 0x0` (no throttle) |
| CPU / RAM | AVX2 only ("this CPU has no AVX-512: the expert kernels run on AVX-2"), 17 pool workers, 128 GB RAM |
| Model | Qwen3.8-Flash-Next native **IQ3_S** pack, 24,576-expert profile, PLE Q8_0 via `--ple-gguf`, n_ctx 268288 |
| Engine | local source build, `archs [89]`, CUDA 13, single stream (no `parallel`), 0.1.42 built from `v0.1.42`, 0.1.43 built from `v0.1.43` |
| Args | `--prefill 8192 --vram-reserve-mib 1024 --conversation-cache-mib 24576 --conversation-cache-slots 8 --conversation-cache-min-free-mib 8192 --prompt-cache 28 --spec 6 --spec-min-p 0.5 --mtp mtp/rt --ple-gguf <Q8_0 PLE> --resident-experts` |
| Expert cache | 0.1.42: 18489 slots / 35.21 GiB - 0.1.43: 18356 slots / 34.95 GiB (= 75.2% of the 24,576 profile pairs), policy PROFILE, no eviction |
| Decode cache | hit **99.1-99.5%**, **0.0% of routed experts read over PCIe**; `route tail skip` active in both versions |

## The numbers

SPEED-Bench `qualitative`, `--scenarios` (one sample per category), 10 categories x 6 tiers, tg 256, one engine restart per arm, 60 samples per run:

| engine | run | AVG decode t/s | MTP accept % | prefill t/s |
|---|---:|---:|---:|---:|
| 0.1.42 | 1 | 110.1 | 66.3 | 4571 |
| 0.1.42 | 2 | 114.3 | 70.3 | 4533 |
| 0.1.42 | 3 | 111.9 | 71.5 | 4480 |
| 0.1.42 | 4 | 112.7 | 70.4 | 4543 |
| **0.1.43** | 1 | 108.4 | 70.3 | 4430 |
| **0.1.43** | 2 | **108.6** | 69.6 | 4528 |

- 0.1.42 mean **112.3** with a same-version spread of **2.1 t/s (+-1.9%)** and accept 66.3-71.5 (5.2 pp); 0.1.43 mean **108.5**, spread 0.2.
- **-3.8 t/s = -3.4%** against the 0.1.42 mean; **-3.6% (39/60 samples slower)** against run 4 and **-3.7% (43/60)** against 0.1.43 run 1 vs run 4's arm. Both 0.1.43 runs are below all but the single best 0.1.42 run.
- **Accept rate identical** (69.6% vs 70.0% mean) -> the loss is the per-token decode path, not MTP.
- **Prefill flat**: 4532 -> 4479 (-1.2%), consistent with "longer prompts are unchanged".
- Single tier x category cells are noisy (+-10 t/s run to run: 8k reads 126.6 then 114.6 across the two 0.1.43 runs; 128k reads 100.7 then 118.3), so only the 60-sample average is a signal.

0.1.42 (run 4, the arm paired with the 0.1.43 run below):

```
  tier    code   human    math   multi      qa     rag  reason    role    stem   write     AVG  |     pre_s  pre_tps
  ----------------------------------------------------------------------------------------------------------------------
      8k   120.3   151.1   111.3    95.3   114.0   140.7   113.3   109.0   109.7   100.2   116.5  |    1.8     4363
     32k   117.4   110.1   112.7   112.8   105.6   110.0   129.5   153.2   121.9   106.2   117.9  |    6.8     4637
     64k   106.1   113.0   111.7   101.5   107.4   132.6   135.5   115.5   114.5   107.2   114.5  |   13.4     4688
    128k   101.5   118.3   116.5   104.9   101.0   116.0   127.4   114.7   101.9   108.4   111.1  |   27.1     4628
    200k   116.7   111.8   121.4   101.0   109.7   104.4   105.2   119.3   116.1    97.5   110.3  |   43.5     4508
    256k    94.7   105.8   110.6    92.7   101.4    96.2   130.7   114.7   106.5   102.8   105.6  |   56.6     4435
  ----------------------------------------------------------------------------------------------------------------------
    AVG   109.5   118.3   114.0   101.4   106.5   116.7   123.6   121.1   111.8   103.7   112.7
```

0.1.43 (run 2) - same 60 samples:

```
  tier    code   human    math   multi      qa     rag  reason    role    stem   write     AVG  |     pre_s  pre_tps
  ----------------------------------------------------------------------------------------------------------------------
      8k   104.7   120.6   118.3    97.6    97.2   109.8   135.0   132.7   122.2   108.3   114.6  |    1.8     4327
     32k   109.0   109.4   108.5    97.1   107.6   108.4   121.1   116.1   108.1   111.1   109.6  |    6.7     4668
     64k   106.0   111.7   108.3   114.9    95.0    94.6   119.3   108.1    99.3   103.1   106.0  |   13.4     4676
    128k   121.8   125.9   116.8   100.4   108.6   137.2   114.8   129.2   124.3   104.4   118.3  |   27.0     4638
    200k    99.2   114.7   114.2   102.6    97.9    95.9   122.3   104.4   107.8   100.0   105.9  |   43.3     4523
    256k    87.5    94.5    96.0    84.2    87.5    94.9   113.2   106.4   107.1   100.0    97.1  |   57.8     4335
  ----------------------------------------------------------------------------------------------------------------------
    AVG   104.7   112.8   110.3    99.5    99.0   106.8   121.0   116.1   111.5   104.5   108.6
```

per-cell delta (0.1.43 - 0.1.42), t/s:

```
  tier         code     human      math     multi        qa       rag    reason      role      stem     write   tier avg
  ------------------------------------------------------------------------------------
      8k       -15.6     -30.5      +7.0      +2.3     -16.8     -30.9     +21.7     +23.7     +12.5      +8.1      -1.9
     32k        -8.4      -0.7      -4.2     -15.7      +2.0      -1.6      -8.4     -37.1     -13.8      +4.9      -8.3
     64k        -0.1      -1.3      -3.4     +13.4     -12.4     -38.0     -16.2      -7.4     -15.2      -4.1      -8.5
    128k       +20.3      +7.6      +0.3      -4.5      +7.6     +21.2     -12.6     +14.5     +22.4      -4.0      +7.3
    200k       -17.5      +2.9      -7.2      +1.6     -11.8      -8.5     +17.1     -14.9      -8.3      +2.5      -4.4
    256k        -7.2     -11.3     -14.6      -8.5     -13.9      -1.3     -17.5      -8.3      +0.6      -2.8      -8.5
```

## What I already ruled out on this box

- `STRATA_MMVQ_IL=0` - measured neutral ("basically no difference"), reverted. sm_89 has no measured interleaved table in 0.1.43 (`native_mmvq.cu:2554`: "sm_89 (Ada) has none yet: nobody measured it ... it takes the shared table", measured on RTX 3060 + RTX 5070); `mmvq_il_parity` says the output is bitwise identical either way, so a table is only speed.
- `--vram-reserve-mib 1024 -> 700` - 1.958 MiB/slot, so +165 slots, 75.2% -> 75.9% of a profile with 99.5% hit: no decode change.
- `--prefill` above 8192 - prefill is flat from 8k to 256k contexts (4327-4676 t/s), so chunk size is not the limiter.
- `route tail skip` - active in both (195,149 missed experts skipped); the >=90% gate of #1884 is not reached at 75.2%.
- **`ADAPT_OVERLAP`** (`src/program/generate.cpp:1330`, default-on for Linux CUDA, +5.5-8.3% on an A4000, +4.4% on a P100): unreachable here twice. `generate.cpp:8294` reports "not in the resident RAM mode unless `STRATA_ADAPT_OVERLAP_RESIDENT=1`" for `--resident-experts`, and there is nothing to hide anyway - **0.0% of routed experts cross PCIe during decode**. So the one 0.1.43 decode improvement contributes 0 here while the rest of the release costs 3.4%.

## Ask

1. Is an sm_89 / 4090 interleaved-kernel table planned, since the shared table was measured on 3060 + 5070?
2. Should `ADAPT_OVERLAP` (and whatever else in 0.1.43 sits on the decode path) also be gated off when the decode cache hit rate is ~100% with 0% PCIe traffic, so resident-mode configs do not pay the cost without the benefit?
3. I can bisect the 0.1.42..0.1.43 decode path on this card: each arm is a 3-5 min rebuild + a 12 min 60-sample run, and I will post the table. Say which commits to try.

Bench harness: SPEED-Bench `qualitative.parquet` via a plain OpenAI-compatible client (`/v1/chat/completions`, temperature 0, tg 256, `--runs 1`); decode tps from the engine's own `timings` (`decode_tps`, `draft_n`, `draft_n_accepted`), no client-side estimate. Full per-sample JSONs available on request.