贡献 / #2207

#2207 sycl/A770: wide MMVQ kernels at 2-8 columns in large-GRF mode on Arc A-series (bit-identical; 1.3-7.4x kernel, decode +10-17%)

open · @aslater3 · 0 评论 · 去 GitHub 看

BenchmarksNVIDIA / CUDAModels & quantsLinux

说明

## Title
Issue: #1545, #1606 (Arc A-series decode)

## Summary
The wide MMVQ kernels (Q6_K `a2` and the generic `native_mmvq_wide`) spill registers at 2-8 columns with the default 128 GRFs on an Arc A770, and their bandwidth collapses as the column count grows (Q6_K 2560x10240: 248 GB/s at 1 column, 74 GB/s at 4, 8 GB/s at 8). On Alchemist they now launch in large-GRF mode (256) at 2-8 columns: **1.3-7.4x faster at 2-8 columns, bit-identical**, decode +10-17% on an A770 (the verify window runs these at 2-6 columns every round).

Large GRF halves the threads resident per Xe core, so it is the default on Alchemist only (`strata::intel_gpu_gen`). Battlemage and every other device, and 1 column everywhere, launch the same kernels as today. `STRATA_MMVQ_LARGE_GRF=1` forces it on, `=0` off.

## What changed
- `sycl/src/kernels/cuda/native_mmvq.dp.cpp`: `mmvq_large_grf(queue)` (once per process, Alchemist by default, env override); in `launch_q6k_wide` (a2 path) and `launch_wide`, `if constexpr (NCOLS > 1)` + the runtime switch launch a `grf_size<256>` copy of the same kernel, else the existing launch, unchanged.
- `sycl/src/kernels/mmvq_bench.cpp`: prints a checksum of the output (FNV-1a over the float bits), so the modes can be compared bit for bit.

## How it was checked
Arc A770 16 GB, `xe`, Linux 7.0, oneAPI 2026.1 (JIT), Ryzen 7 3800X, 30 GB RAM, same day.

`mmvq_bench <n_in> <n_out> <cols>`, Q6_K, `main` vs this PR (us; checksums identical with `STRATA_MMVQ_LARGE_GRF=0` and `=1` in all 15 cases):

| shape | 1 col | 2 cols | 4 cols | 6 cols | 8 cols |
|---|---|---|---|---|---|
| 2560 x 2560 | 33.5 -> 33.4 | 55.2 -> 30.0 | 97.0 -> 36.0 | 150.7 -> 44.5 | 631.1 -> 153.1 |
| 2560 x 10240 | 86.6 -> 86.1 | 157.4 -> 79.4 | 289.3 -> 92.9 | 552.8 -> 110.2 | 2726.5 -> 370.5 |
| 10240 x 2560 | 83.7 -> 83.9 | 119.4 -> 88.9 | 237.4 -> 103.1 | 393.9 -> 129.8 | 1704.4 -> 465.2 |

End to end, `a0116b27` + #2190 + #2193 + #2203 with and without this PR, Coder IQ1_M, 8K-token prompt, 256 tokens, greedy, `--resident-experts --max-context 131072 --kv int8 --kv-resident 20480 --spec 4 --mtp`, `STRATA_VERIFY_STEPPED=1`:
- decode 23.47 / 23.50 / 23.73 / 23.77 -> 25.99 / 27.10 / 27.49 tok/s
- prompt unchanged (472-502 tok/s both, within run-to-run spread)
- output ids identical (`72b762d5b2a3`) in every run with the same expert placement; one run had a different cache placement (hits 0.7605 vs 0.7457-0.7468) and different ids, the CPU-vs-GPU expert rounding seen on `main` too.

Not built or run on Battlemage or i915. SYCL-only files.

## Extra Notes
Several other runs did not finish because of the decode hang in #2194 (both builds); they are left out.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

本站相关内容

相关页面的快捷入口。