Pull requests / #2150
#2150 SYCL: opt in to 256 GRFs for fused hyperconnection up
open · @alex4o · 0 comentarios · En GitHub
BenchmarksAMD / HIPNVIDIA / CUDAModels & quants
Descripción
## Summary Add an opt-in 256-GRF launch for the default fused hyperconnection up kernel. On an Intel Arc A770 16 GB / i915 with oneAPI 2026.1, this removed the measured register-spill allocation and improved matched warmed decode by about 13%. Related: #1606 (decode optimization only; this does not claim to fix that report's prefill bottleneck). ## What changed Only sycl/src/kernels/cuda/fused_gr.dp.cpp changes: - STRATA_GR_UP_LARGE_GRF=1 selects grf_size<256> and a distinct kernel identity. - Unset, 0, or any value other than 1 retains the original kernel name and launch properties. - Body, arithmetic, indexing, range, subgroup size 32, and root-sync property are shared/unchanged. Other GR paths are untouched. - The flag is sampled once: restart the engine to change it. The option remains off by default. The experimental Intel GRF header is an unconditional compile dependency; validation is on oneAPI 2026.1, the toolchain documented for this port. Older compilers and other GPUs were not tested. ## Validation **This PR on base a0116b2:** full SYCL strata and gr_bench builds passed in strata-sycl-dev with icpx 2026.1. Separate gr_bench runs printed effective flag values unset / 0 / 1; all six output checksums (T=1..6) matched each other and the earlier OFF checksums. CUDA/HIP source is unchanged. **Earlier full-model experiments on base 61b3fb5, with this exact kernel patch:** | Matched OFF decode | Matched ON decode | |---:|---:| | 18.7 tokens/s | 21.2 tokens/s | | 18.6 tokens/s | 21.1 tokens/s | | 19.3 tokens/s | 21.4 tokens/s | Ryzen 5800X, 48 GB-class RAM, IQ3_XXS, INT8 KV, 131072 capacity, 4096 prefill, spec4, CPU cache misses. Two warm-ups per arm; three 384-token timing samples without prefix reuse. Median gain 13.4%. These capped samples are not complete-answer correctness tests. - Compiler metadata: 3520-byte spill allocation -> zero; 256 GRFs. Private-memory size remains 40192 bytes. - A direct fused-kernel fixture covered 432 cases / 77,575,968 output floats, including T=1..8, injection/apply/in-place variants. OFF/ON and repeated OFF were bit-identical; finite/guard checks passed. - A 129,845-token prompt and complete 549-token OFF reference vs exact-token ON replay produced 136,327,680 bit-identical float32 logits. Both arms used the same isolated diagnostic capture/teardown fix and controlled T1/adaptation-off execution; neither diagnostic is part of this PR. - An independent unforced ON run with production speculation/adaptation emitted the same complete answer. OFF and ON recovered 5/5 codes and passed 384 LRU checks each. - A later OFF/OFF repeat showed numerical variation. The matched equality is evidence for this tested pair, not a claim of deterministic repetition or universal numerical equivalence. [Detailed protocol, limitations and reproducible direct fixture](https://gist.github.com/alex4o/6f5912c82d67393fe5ca3aac26ff5b63). Full-model speed/128K tests were not rerun on current upstream a0116b2; the changed GRF source was unchanged upstream between the two bases. No universal 20+ tokens/s claim.
En el sitio
Enlaces a install, modelos, releases.