Pull requests / #2586
#2586 sycl/A770: decode GDN recurrence in large GRF, and the split recurrence by default on Alchemist (bitwise; recurrence 2.6x faster)
open · @aslater3 · 0 commentaires · Sur GitHub
NVIDIA / CUDAModels & quantsWindows
Description
## Title sycl/A770: decode GDN recurrence in large GRF, and the split recurrence by default on Alchemist (bitwise; recurrence 2.6x faster) ## Summary On an Arc A770 the decode window's GDN recurrence kernels spilled registers: the step kernel used ~7.6 KB of scratch per SIMD32 thread at the default 128 registers, according to unitrace's compile table. Its 32 state rows per thread alone fill the default register file. On Alchemist this PR does two things: - launches these kernels in large-GRF mode (256 registers per thread); - makes upstream's split recurrence (S25/S26, `STRATA_GDN_SPLIT`, already bitwise) the default. The arithmetic is unchanged, so the output is bit-identical. Every other device launches the same kernels as before. ## What changed - `sycl/src/kernels/cuda/verify_kernels.dp.cpp`: - `gdn_large_grf()`: Alchemist by default; `STRATA_GDN_LARGE_GRF=0/1` overrides. - `gdn_launch<Name>()`, which picks the large-GRF or default launch for every kernel `gdn_step_norm_multi` launches. - `STRATA_GDN_SPLIT` now defaults to on for Alchemist (`=0` / `=1` still choose). - `sycl/src/kernels/gdn_bench.cpp` (new, plus a `CMakeLists.txt` target): `gdn_step_norm_multi` at the Coder's shape (h_k 16, h_v 48, S 128) for 1-6 tokens, with 48 layers' own states so they stream from VRAM, host-timed, with an output checksum. ## Extra Notes Arc A770 16 GB, xe, compute runtime 26.05, oneAPI 2026.1. `gdn_bench` timings: | tokens | before (no split) | split | split + large GRF | |---|---|---|---| | 1 | 68.8 us | 79.2 | 42.0 | | 4 | 95.0 | 103.1 | **39.5** | | 6 | 115.8 | 114.2 | **44.9** | - **Correctness:** `gdn_parity` reports 0 failures, including `gdn_step_norm_multi T=1..8 == single-token`, bitwise in all 47 cases. - **Engine:** Coder IQ1_M, 8K prompt, `--resident-experts`, stepped window, same-day pairs. The verify window's GPU wait per round went 39.5 / 38.8 -> 37.6 / 37.9 ms with the split, then 35.9-36.3 -> 33.8-34.0 ms with large GRF on top (3 pairs). The output ids follow the usual run-to-run CPU/GPU placement variation. Not tested: A750 / A580, i915, Windows. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Sur le site
Liens install, modèles, releases.