Pull requests / #2199
#2199 sycl: prompt GDN recurrence with one barrier per token on Alchemist (bit-identical; A770 kernel 2.1x, prompt +3-7%)
open · @aslater3 · 0 コメント · GitHub で見る
BenchmarksAMD / HIPNVIDIA / CUDAModels & quantsDocumentationLinux
本文
## Title Issue: #1545, #1606 (Arc A-series prompt speed) ## Summary The prompt path's GDN recurrence on Alchemist: the column kernel needs five work-group barriers per token; this variant keeps its four row groups and FMA chains **in the same order**, but puts four columns x four row groups in one 16-lane sub-group, so the two cross-row-group sums become sub-group shuffles and a token needs one barrier. Bit-identical output; ~2.1x faster kernel, +3-7% prompt speed on an Arc A770. On by default on Alchemist only (`strata::intel_gpu_gen`); every other card keeps the column kernels, unchanged. `STRATA_GDN_XE=0` turns it off, `STRATA_GDN_XE=1` turns it on for any Intel GPU with 16-lane sub-groups (a reviewer with a B-series can check bits and time with it). ## What changed - `sycl/include/strata/prefill/gdn_xe.hpp` (new): the kernel (variant 6 of `gdn_recurrence_variant`, 256 GRFs). - `sycl/src/prefill/kernels.dp.cpp`: the gate (`gdn_xe_on`, once per process) and the dispatch; `STRATA_GDN_HEAD` / `STRATA_GDN_REC_HEADS` still win. - `include/strata/prefill/kernels.hpp`: the variant's comment. - `sycl/src/kernels/gdn_parity.cpp`: a prefill section comparing variant 6 with variant 0 bit for bit (state, FP32 and FP16 outputs; T = 1, 7, 256, 1000, two seeds, the production entry point and T = 0); skips on anything that is not an Intel GPU with 16- and 32-lane sub-groups. `--prefill-tokens N --prefill-repeat R` times both kernels. - `sycl/CMakeLists.txt`: links `gdn_parity` to `strata_prefill`; `ctest -R prefill_gdn_parity`. - `docs/INTEL.md`: one bullet in the Alchemist section. ## How it was checked Arc A770 16 GB, `xe`, Linux 7.0, oneAPI 2026.1 (JIT), Ryzen 7 3800X, 30 GB RAM. Same day, `a0116b27` + #2190 + #2193 (without #2190 this card cannot start; #2193 removes a decode stall that otherwise dominates every run) with and without this commit. - `gdn_parity --selftest --prefill-only`: 0 failures. Timing, 48 heads, 0 mismatches, max_abs 0: | tokens | column kernels | this PR | |---|---|---| | 8,192 | 51.0 ms | 24.4-24.6 ms | | 32,768 | 203.6-235.9 ms | 110.0-112.6 ms | - End to end, Coder IQ1_M, `--resident-experts --expert-cache auto --max-context 131072 --kv int8 --kv-resident 20480 --vram-reserve-mib 1200 --prefill-borrow --prefill auto --spec 4 --mtp`, greedy: | prompt | without | with this PR | |---|---|---| | 8K tokens | 386.4 tok/s | 396.3, 406.7 tok/s | | 32K tokens | 477.9, 480.0 tok/s | 511.1, 517.5 tok/s | Output ids identical to the baseline in every run (8K `72b762d5b2a3`, 32K `eb3432fea981`). Not run on Battlemage or i915. CUDA/HIP: SYCL-only files (plus a comment in a shared header). ## Extra Notes Tried and not taken: the same kernel with its state in SLM (no register spills, same bits, ~4x slower: 13.7 vs 3.1-3.3 ms at T=1000). 🤖 Generated with [Claude Code](https://claude.com/claude-code)
関連リンク
インストール・モデル・リリースへの站内リンク。