Pull requests / #1711

#1711 sycl: opt-in grouped XMX expert GEMM for the prompt path (STRATA_PF_XMX), with a parity test

open · @aslater3 · 0 コメント · GitHub で見る

AMD / HIPNVIDIA / CUDAModels & quantsLinux

本文

Adds an opt-in grouped XMX GEMM for the prompt path's routed experts on Intel Arc. With `STRATA_PF_XMX` unset, the default path does not change.

## What it changes

- `STRATA_PF_XMX=1`: dequantizes a group of up to 16 routed experts to FP16, then runs one `joint_matrix` launch per product. A standalone measurement against oneMKL on uneven routing: down 1.75–6x faster, gate/up 0.8–3x. Above `STRATA_PF_XMX_GU_MAX=250` tokens per expert, gate/up stays on oneMKL.
- `STRATA_PF_XMX=2`: the same GEMM, but the kernel decodes the IQ3_XXS / IQ3_S / IQ2_S / IQ4_XS gate/up rows and the IQ4_NL / Q2_0 down rows itself, so there is no FP16 staging pass.
- New `xmx_group_parity` test (built with `STRATA_SYCL_PARITY=ON`): checks every supported type against dequant+oneMKL.

The second commit fixes the in-kernel decode after rebasing onto current `main`. `main` now has the IQ2/IQ3/IQ4_NL/IQ1_M decoders write values `[8·tid, 8·tid+8)` in coalesced runs. The grouped kernel was still passing the old `sub*8 + o32/32` thread index, so four of the six types decoded garbage (relative error ≈10). It now passes `tid = o >> 3` for every type except IQ4_XS, which keeps its own layout.

## Measured

On an Arc A770 16 GB (xe driver, oneAPI 2025.3, Linux 7.0), `xmx_group_parity` on this branch:

```
type 18 gate/up rows 1365: dequant+oneMKL 2.593 ms | fused xmx 1.800 ms (1.44x) relerr 9.9e-07 OK
type 21 gate/up rows 1365: dequant+oneMKL 1.507 ms | fused xmx 1.765 ms (0.85x) relerr 1.1e-06 OK
type 22 gate/up rows 1365: dequant+oneMKL 1.238 ms | fused xmx 1.330 ms (0.93x) relerr 1.5e-06 OK
type 23 gate/up rows 1365: dequant+oneMKL 2.498 ms | fused xmx 1.516 ms (1.65x) relerr 1.8e-06 OK
type 20 down    rows 1365: dequant+oneMKL 0.825 ms | fused xmx 0.773 ms (1.07x) relerr 4.6e-07 OK
type 42 down    rows 1365: dequant+oneMKL 0.700 ms | fused xmx 0.603 ms (1.16x) relerr 1.5e-07 OK
all OK
```

Without the second commit, types 18, 21, 22 and 20 fail with relative error ≈10.

End to end, on the Coder IQ1_M model with an 8K prompt:
- Earlier local measurements, before the rebase: 0.5–3K-token prompts were 14–29% faster with `STRATA_PF_XMX=2`, and 8K/32K prompts 2–5% faster. These were not repeated on this branch.
- This branch, `main` + #1602 on the A770: a same-day 8K run did not show a gain.

So it stays opt-in.

## Not tested

- B-series (Battlemage) and other Alchemist cards.
- The CUDA and HIP builds: these files are SYCL only (`sycl/`), plus one declaration in `include/strata/kernels/iq_kernels.hpp` and a new `include/strata/prefill/xmx_moe.hpp`, neither of which the CUDA/HIP sources include.
- A full logit comparison end to end with `STRATA_PF_XMX=2`. The parity test covers the kernel only.

関連リンク

インストール・モデル・リリースへの站内リンク。