Pull requests / #2199

#2199 sycl: prompt GDN recurrence with one barrier per token on Alchemist (bit-identical; A770 kernel 2.1x, prompt +3-7%)

open · @aslater3 · 0 commentaires · Sur GitHub

BenchmarksAMD / HIPNVIDIA / CUDAModels & quantsDocumentationLinux

Description

## Title
Issue: #1545, #1606 (Arc A-series prompt speed)

## Summary
The prompt path's GDN recurrence on Alchemist: the column kernel needs five work-group barriers per token; this variant keeps its four row groups and FMA chains **in the same order**, but puts four columns x four row groups in one 16-lane sub-group, so the two cross-row-group sums become sub-group shuffles and a token needs one barrier. Bit-identical output; ~2.1x faster kernel, +3-7% prompt speed on an Arc A770.

On by default on Alchemist only (`strata::intel_gpu_gen`); every other card keeps the column kernels, unchanged. `STRATA_GDN_XE=0` turns it off, `STRATA_GDN_XE=1` turns it on for any Intel GPU with 16-lane sub-groups (a reviewer with a B-series can check bits and time with it).

## What changed
- `sycl/include/strata/prefill/gdn_xe.hpp` (new): the kernel (variant 6 of `gdn_recurrence_variant`, 256 GRFs).
- `sycl/src/prefill/kernels.dp.cpp`: the gate (`gdn_xe_on`, once per process) and the dispatch; `STRATA_GDN_HEAD` / `STRATA_GDN_REC_HEADS` still win.
- `include/strata/prefill/kernels.hpp`: the variant's comment.
- `sycl/src/kernels/gdn_parity.cpp`: a prefill section comparing variant 6 with variant 0 bit for bit (state, FP32 and FP16 outputs; T = 1, 7, 256, 1000, two seeds, the production entry point and T = 0); skips on anything that is not an Intel GPU with 16- and 32-lane sub-groups. `--prefill-tokens N --prefill-repeat R` times both kernels.
- `sycl/CMakeLists.txt`: links `gdn_parity` to `strata_prefill`; `ctest -R prefill_gdn_parity`.
- `docs/INTEL.md`: one bullet in the Alchemist section.

## How it was checked
Arc A770 16 GB, `xe`, Linux 7.0, oneAPI 2026.1 (JIT), Ryzen 7 3800X, 30 GB RAM. Same day, `a0116b27` + #2190 + #2193 (without #2190 this card cannot start; #2193 removes a decode stall that otherwise dominates every run) with and without this commit.

- `gdn_parity --selftest --prefill-only`: 0 failures. Timing, 48 heads, 0 mismatches, max_abs 0:

  | tokens | column kernels | this PR |
  |---|---|---|
  | 8,192 | 51.0 ms | 24.4-24.6 ms |
  | 32,768 | 203.6-235.9 ms | 110.0-112.6 ms |

- End to end, Coder IQ1_M, `--resident-experts --expert-cache auto --max-context 131072 --kv int8 --kv-resident 20480 --vram-reserve-mib 1200 --prefill-borrow --prefill auto --spec 4 --mtp`, greedy:

  | prompt | without | with this PR |
  |---|---|---|
  | 8K tokens | 386.4 tok/s | 396.3, 406.7 tok/s |
  | 32K tokens | 477.9, 480.0 tok/s | 511.1, 517.5 tok/s |

  Output ids identical to the baseline in every run (8K `72b762d5b2a3`, 32K `eb3432fea981`).

Not run on Battlemage or i915. CUDA/HIP: SYCL-only files (plus a comment in a shared header).

## Extra Notes
Tried and not taken: the same kernel with its state in SLM (no register spills, same bits, ~4x slower: 13.7 vs 3.1-3.3 ms at T=1000).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Sur le site

Liens install, modèles, releases.