#3454 [Engine] Scope: lossless 4-bit slot layout for resident IQ2_S / IQ3_XXS / IQ3_S gate/up experts (opt-in, Pascal GP100 first)

Descrição

## Problem

On a GP100 die (Tesla P100, PH402; sm_60, no `__dp4a`) the routed experts' gate/up kernels for the i-quant formats are
nowhere near the memory wall. Measured in isolation on one die (T = 1, 10 experts per token):

| format | bytes per expert | time per expert | effective bandwidth | share of 732 GB/s |
|---|---:|---:|---:|---:|
| IQ2_S | 1.05 MB | 10.18 us | ~103 GB/s | 14% |
| IQ3_XXS | 1.25 MB | 9.96 us | ~126 GB/s | 17% |
| IQ3_S | 1.41 MB | 12.47 us | ~113 GB/s | 15% |

The time is the per-block serial chain (wait for the weights, sync, unpack the codebook indices from the packed grid,
the emulated dp4a, the 16-lane reduction), not the bytes. Removing every dot product from the kernel still leaves
about two thirds of its time. So a layout that costs more bytes but fewer instructions per block wins on this chip.

## Proposed behavior

`STRATA_EXPERT_NIBBLE=1` (opt-in; unset = today's engine, byte-identical):

- At load, for every layer whose routed experts are all resident in a card's cache, the IQ2_S / IQ3_XXS / IQ3_S
  gate/up blocks are transcoded into a slot layout of 4-bit codes (the codebook indices stored one per nibble, the
  scales kept as they are). The down projections are untouched. The transcode is checked by a round trip back to the
  GGUF bytes, and every slot carries a hash that the engine re-checks (`STRATA_EXPERT_NIBBLE_RECHECK=1`).
- The decode (verify window) gate/up kernels and the prompt path's gate/up kernels for those layers read the 4-bit
  slots. The arithmetic order is the production kernels' order, so the outputs are bit for bit the same.
- Layers of other formats (IQ2_XS, IQ2_XXS, ...) and layers with experts outside the cache keep today's path. Options
  that read or write the resident experts as GGUF bytes refuse the switch at start-up with a message instead of
  silently running the old kernels.

Cost: the 4-bit slots take about 1.7x the GGUF bytes of those layers. On a 4-die layer split of Swift 1.5 IQ3_S (47
converted layers) that is +4.0 GiB per die; on the IQ3_XXS pack (29 of 48 layers convert) +7.1 GiB in total. Load time
+0.3 s.

## Evidence (measured on main e88ee39d / engine 0.1.45.3 plus the change; see the pull request)

**Environment note: all of it with every expert resident in VRAM** (four GP100 dies, 64 GB in all; no CPU or PCIe share, no adaptive tier). The switch refuses a layer that is not fully resident, so a card with experts partly in RAM keeps today's path.

Decode of eight prompts, token-weighted, the unchanged engine against the same engine with the switch on, one fresh engine per arm: 47.3 -> 51.9 tok/s (+9.7%, each prompt +9.5..+10.9%); prompt reading 2.3..3.0% slower; 15 of 15 replies bit for bit the unchanged engine's; +12.7 GiB of VRAM over the four dies.

## Earlier evidence on our 0.1.41-based tree

2x PH402 SKU 200 (4x GP100, 16 GB HBM2 each, 48 SMs, clocks locked at 715 MHz memory / 1050 MHz core), Windows 11,
driver 581.80 (TCC), CUDA 12.9 build with `STRATA_EXPERIMENTAL_SM60`, layer split 12,24,36, `--pipeline-windows 2`,
MTP with `--spec-min-p 0.85`, 1M context, all experts resident, one fresh engine per arm.

- Isolated gate/up kernel, 512 experts per layer, T = 1..5, both routing extremes: 460.8 M output rows compared, 0
  different; gate/up time x1.45..x1.58 (T = 1 floor x1.43), 2.5..3.65 ms saved per decode window.
- Swift 1.5 IQ3_S, 8 prompts (code, SQL, story, explanation; 8K read; 26K read + 400; 88K append + 400):
  decode token-weighted 52.02 -> 55.91 tok/s (+7.5%; each prompt +7.2..+8.7%); prompt reading 1.5..2.5% faster at
  1K..30K; 15 of 15 replies bit-identical to the switch-off arm (including a 260K-token read), the engine's state
  hashes identical, every slot's hash identical after each request. Peak VRAM +4.0 GiB per die.
- Swift 1.5 IQ3_XXS: 14 of 14 replies and state hashes identical; decode +4.5..+5.8%.
- Prompt path (IQ2_S layers): 198 comparisons, 2,872,320 rows, 0 different; 10..16% faster at <= 10 tokens per
  expert, 2.6..5% faster at >= 25.

## Tradeoffs and limits

- VRAM: the slots are extra bytes on top of the resident cache; cards that are full will not fit them.
- Only the three formats; only layers that are fully resident; nothing changes for CPU or PCIe-share experts.
- The kernels are written and tuned for sm_60 (no tensor cores, no dp4a). They compile for any CUDA arch, but we have
  measured them on GP100 only. We propose to gate the call sites under `STRATA_EXPERIMENTAL_SM60`, like #1097, so the
  default engines do not carry them until someone measures another card.
- About 2,200 lines of engine code plus a parity/bench tool.

## Questions for the maintainers

1. Is the scope acceptable as one opt-in PR (loader transcode + decode kernels + prompt-path kernels + parity tool)?
2. Gated under `STRATA_EXPERIMENTAL_SM60`, or compiled everywhere and opt-in by the switch?
3. We can validate on 4x GP100 only. A single P100 run by you with the identity gate would be welcome.