#2069 sm_75: Turing tables for the decode matvecs and the grouped experts (bitwise, decode +6 %)

Descrição

Two Turing-only defaults, both bitwise equal to what RTX 20 runs today, both with a switch to turn them off. Measured on an RTX 2080 Ti; I have no other sm_75 card to check them on.

1. **`kIlRows_75`** (`native_mmvq.cu`): the interleaved small-batch matvec kernels with their own table, as Volta and Pascal have. The 2-6 column dense projections of a verify window ran the plain multi-column kernels, which on Turing are bound by `dp4a` throughput from 2 columns on. `STRATA_SM75_IL=0` keeps the plain kernels. No one-column table: the engine's dedicated single-column kernels sum in another order, for a 0.5 % gain.
2. **Expert layout mode 4 on sm_75** (`iq_kernels.cu`, `exp_mode()`): the AMD layouts already build for CUDA; mode 4 is bitwise equal to mode 0 and faster for the window's usual 1-2 entries per expert. Its down projection loses once groups hold 3+ entries, so windows of 4+ tokens (`cap_groups > 30`) keep mode 0 for the down projection. `STRATA_SM75_EXP=0` keeps mode 0; `STRATA_EXP_MODE` still overrides.

## Setup

RTX 2080 Ti 22 GB (sm_75, PCIe 3.0 x16), i7-6700, 56 GB DDR4-2133, CUDA 13.0, driver 595, SM clock locked at 1650 MHz, 240 W. `v0.1.43` built from source with `-DSTRATA_BUILD_TESTS=ON`. Model Q2_0, `--resident-experts`, `--spec 4 --spec-min-p 0.7`, `--max-context 524288` (YaRN ×2), `--kv int8 --kv-resident 32768`.

## 1. The matvec table: `mmvq_il_parity --bench --emit-table`

Bitwise equal to `native_mmvq` (the parity run). Selected rows (µs; "il" = the interleaved kernel with the rows per warp the table picks):

| shape | columns | plain | il |
|---|---:|---:|---:|
| IQ4_XS 2560×10240 | 2 / 4 | 58.8 / 81.1 | 32.8 / 37.4 |
| Q4_K 2560×10240 | 2 / 4 | 45.3 / 74.5 | 32.8 / 39.1 |
| Q5_K 2560×12288 | 2 / 4 | 56.9 / 88.1 | 43.9 / 56.9 |
| Q6_K 2560×12288 | 3 / 4 | 77.3 / 95.0 | 55.5 / 58.7 |
| Q5_K head 2560×248320 | 2 / 4 | 1047 / 1671 | 782 / 1045 |

The 3 % rule of the other tables; the full `--emit-table` output is in the commit's table.

## 2. Expert layout: `native_expert_bench <shard1> 20,40 G T 0 4`

Real Q2_0 rows of Flash-Next, bitwise equal to mode 0 in every case (`0 differing`):

| groups × entries | mode 0 | mode 4 | gate/up | down |
|---|---:|---:|---|---|
| 10 × 1 | 47.3 µs | 41.4 µs | 28.3 → 25.8 | 17.1 → 13.4 |
| 20 × 1 | 83.7 | 71.5 | 49.6 → 43.5 | 33.4 → 27.8 |
| 30 × 1 | 120.2 | 100.2 | 71.5 → 61.2 | 48.1 → 38.9 |
| 15 × 2 | 96.2 | 87.0 | 59.3 → 51.3 | 36.2 → 34.7 |
| 12 × 3 | 102.4 | 102.2 | 65.1 → 59.7 | 36.5 → **40.8** |
| 16 × 4 | 161.1 | 165.0 | 103.2 → 95.4 | 56.9 → **68.9** |

The bold rows are why the down projection keeps mode 0 in windows of 4+ tokens.

## Engine: same binary, switches off / on

One-shot, 256 new tokens, greedy, runs interleaved off / on / on / off (off = `STRATA_SM75_IL=0 STRATA_SM75_EXP=0`), decode tok/s:

| prompt | off | on | |
|---|---:|---:|---:|
| 4K | 52.35 / 52.45 | 55.79 / 55.74 | +6.5 % |
| 32K | 50.72 / 50.14 | 53.52 / 53.16 | +5.8 % |

At 4K all four runs wrote the same tokens (the same MD5 off and on); prefill is unchanged (1012-1016 / 1352-1354 tok/s). In `--serve` steady state (4K prompts, 99 % cache hits) the table alone took ~69 to ~76-77 tok/s.

The token output of a one-shot run here is not fully deterministic run to run (two MD5s appear with any build, from the CPU/GPU split of the experts), so runs are compared within the same MD5.

## Tests

`mmvq_il_parity` bitwise, `native_grouped_parity` 0 failures, `native_expert_bench` 0 differing for every case above.

🤖 Generated with [Claude Code](https://claude.com/claude-code)