#3252 0.1.45 and 0.2.0-beta (engine 0.1.45.3): decode −10% and prefill −20% on an A100 40GB (sm_80); 0.1.44.1 unaffected

Descrição

# 0.1.45 and 0.2.0-beta (engine 0.1.45.3): decode −10 % and prefill −20 % on an A100 40 GB (sm_80); 0.1.44.1 unaffected

**Machine / build**

- NVIDIA **A100-PCIE-40GB (sm_80)**, single GPU, driver 615.78.08; 125 GB RAM; i5-11600K (6C/12T, AVX2 only); Arch Linux.
- Engine built from source for every tag inside Docker with `-DCMAKE_CUDA_ARCHITECTURES=80`: `v0.1.43.1` (2d70b1bf), `v0.1.44.1` (b3e02db), `v0.1.45` (c4074a70), `v0.2.0-beta` (3d1fb48, engine 0.1.45.3).
- Model: **Qwen3.8-Flash-Next GSQ-RCO IQ3_S** — native pack + 2 GGUF shards, PLE in RAM, MTP drafter. Expert cache 14322 slots / 27.2 GiB.
- Engine args byte-identical across arms: `--pack … --expert-profile … --expert-cache auto --prefill auto --spec 4 --spec-min-p 0.5 --mtp …/rt --max-context 262144 --kv int8 --ple-io ram --kv-resident 32768 --vision --vram-reserve-mib 700`.
- One config difference: the 0.1.4x arms carry the experimental-speed-projection control vector; the 0.2.0-beta arm cannot (the cvec launch flags were renamed/removed in 0.1.45.3 — `--cvec-mode project` dies with `unknown argument: project`). ESP measured ±0 % decode in an earlier A/B on this box, so it does not explain the gap.

**Method**

Identical fixed prompts per arm, one request at a time, engine restarted per arm, decode read from the engine's own `timings.predicted_per_second`:

- **tetris probe**: fixed prompt, seed 42, temp 1.0, top_p 0.95, top_k 20, max_tokens 16384 (every arm length-caps at 16384, so wall time is directly comparable).
- **sand probe**: same envelope with a second fixed prompt, also 16384-token runs.
- **prefill**: fixed ~15.1k-token prompt (a repeated pattern with a fresh `[run i]` prefix so nothing is cached), `max_tokens 8`, temp 0, `cache_n = 0`, median of 2 runs, tok/s from `timings.prompt_per_second`.

**Results**

| engine | tetris decode (t/s) | sand decode (t/s) | prefill 15.1k (t/s) |
|---|---:|---:|---:|
| 0.1.43.1 | 111.4 | 106.3 | 3,043 |
| 0.1.44.1 (reference) | 111.3 | 104.9 | 3,056 |
| 0.1.45 | 100.8 (−9 %) | 94.2 (−10 %) | 2,436 (−20 %) |
| 0.2.0-beta / 0.1.45.3 | 100.2 (−10 %) | 71.2 (−32 %) | 2,434 (−20 %) |

The 45-line regression is fully inherited by 0.2.0-beta. In the beta arm the sand run's output also broke format (no fenced HTML, ~10 KB vs the usual ~31 KB) — noted in passing, not measured systematically.

**Suspects**

1. **The MMVQ/IL shared table.** #1842 documented that `native_mmvq_il_rows_for` has only a `kIlRows_70` entry; every sm_80+ card inherits `kIlRowsShared`, measured on an RTX 3060 (sm_86) and RTX 5070 (sm_120). The A100 uses the same guess. Same class of gap as the 0.2.0 release notes' new sm_86/sm_60 decode tables (+10–14 % there) — sm_80 got no measured entry in either case.
2. **The 45-line decode kernel defaults** (route-tail-skip is already the 44.1 default, so the delta vs 44.1 is something newer).
3. **The prefill path** — −20 % with identical args points at the prompt side too. #3204's `f3be9743` warm-up regression is HIP-only; a CUDA-side equivalent or the prefill chunk-path changes are candidates.

**Ruled out on our side**: CUDA arch is explicitly 80 in all four builds (no other GPU in the box); identical pack, args, expert-cache size and VRAM budget across arms; ESP (±0 %); no storage-tier involvement (everything resident, `ple-io ram`).

**What we are asking**

1. Was sm_80 in the identity/perf test set for the 0.1.45 kernel changes? (The release notes list sm_60/sm_86/gfx1201/Intel/M4 — no Ampere.)
2. Which no-rebuild switches should we A/B on 0.2.0-beta to localize it — `STRATA_MMVQ_IL=0`, `STRATA_MMVQ_IL_ROWS=…`, `STRATA_ROUTE_TAIL_SKIP=0`, prefill switches? Happy to run any set on this box and report numbers; all four images stay tagged, and PROD here stays on 0.1.44.1 meanwhile.

*Repro*: `docker build --build-arg CUDA_ARCHITECTURES=80` at each tag; serve with the args above; the three probes are reproducible from the envelopes described under Method (fixed prompts, per-arm fresh engine).