#2441 0.1.44.1: ~20-25% decode regression on RTX A3000 12GB (sm_86), IQ3_XXS GSQ-RCO pack, --spec 4

Beschreibung

# 0.1.44.1: ~20-25% decode regression on RTX A3000 12 GB (sm_86), IQ3_XXS GSQ-RCO pack, `--spec 4`

**Machine / build**

- Laptop, single GPU: **RTX A3000 Laptop 12 GB, sm_86**; i7-12850HX (8P+8E, no AVX-512); 125 GB RAM; CachyOS.
- Engine built from source (Nix, CUDA from nixpkgs), llama.cpp pinned to `3cf03257f219afbe7334045ff7c6a06ac68c627d` (upstream's `third_party/ggml/VERSION.txt`).
- Pack: **GSQ-RCO IQ3_XXS (Swift-Qwen3.8-Flash-Next)**, native pack + MTP draft layer.
- Serve config: `--pack … --native … --ple-gguf … --expert-profile … --expert-cache auto --prefill auto --spec 4 --mtp … --max-context 524288 --kv int8 --kv-resident 20480 --rope-scaling yarn --rope-scale 2 --vram-reserve-mib 1800 --pcie-frac 0.35 --spec-min-p 0.7 --pool-workers 15 --adapt-every 1` (serve + idle-unload; single request at a time).

**Method**

Identical request on both engines on the same machine, back to back: one fixed prompt ("Write a Python module that reads an INI config file … Code only."), `temperature 0`, `max_tokens 300`, 1 warmup + 3 measured runs. Decode tok/s and draft acceptance are read from the engine's own serve log line (`prompt N tokens … M generated in … ms (X tok/s), drafts accepted a of b`). Engines were swapped by pointing the server's `exe` at the two built binaries; no rebuild between runs, no other traffic on the engine.

**Results**

| engine | decode (tok/s) | drafts accepted |
|---|---|---|
| **0.1.43.1** | **38.1, 38.5, 40.2, 42.1** | 78.9%, 81.1%, 84.3%, 85.7% |
| 0.1.44.1 (defaults) | 28.7, 32.0, 35.0, 39.7 | 73.9%, 80.3%, 81.1%, 83.6% |
| 0.1.44.1, `STRATA_ROUTE_TAIL_SKIP=0` | 23.7, 24.0, 24.4, 25.8 | 79.1%, 79.4%, 80.4%, 83.1% |

So 0.1.44.1 is ~20-25% slower to decode here, with or without the new route-tail-skip policy. 0.1.43.1's numbers are the reference we are used to (agent workloads with higher acceptance reach the high 40s).

**Observations that may help**

- In this regime (decode expert-cache hit rate 66-69%, 12 GB card missing lots of experts) `--route-tail-skip` turns **on** by default. It is worth ~+25% *within* 0.1.44.1 (28.7-35 vs 23.7-25.8) but does not reach 0.1.43.1, so it looks like a mitigation of a regression, not its cause.
- Draft acceptance is only part of it: the `=0` arm runs at 79-83% acceptance yet decodes 24 tok/s, below 0.1.43.1's 38-42 at similar acceptance.
- Suspects we have not singled out yet: the new **`STRATA_COMMIT_BATCH`** default (on), and the "**dense Q8_0 matrices read one row per block** … (RDNA4 and **sm_86**)" change from the 0.1.44 notes — sm_86 is exactly this card.
- Relevant log lines on 0.1.44.1: `decode expert cache hit rate: 66.4% / 68.5%`, `route tail skip: 86800 missed experts skipped (90774 entries)`, `KV streaming: 99.8% of block reads hit VRAM`.

Happy to run any A/B you want on this box (we keep both engines built and can switch them deterministically) and report back — e.g. `STRATA_COMMIT_BATCH=0`, an explicit `STRATA_ROUTE_TAIL_SKIP` rank, or a build with the sm_86 dense-Q8_0 row path disabled.