贡献 / #2203

#2203 sycl/A770: fused GR read with more, smaller work-groups on Arc A-series (same sums; decode +3-5%; on top of #2193)

open · @aslater3 · 0 评论 · 去 GitHub 看

BenchmarksNVIDIA / CUDAModels & quantsLinux

说明

## Title
Depends on #2193 (this branch contains its commit; review only the second commit, `sycl/A770: fused GR read with more, smaller work-groups on Arc A-series`).

## Summary
After #2193 the fused GR read on an Arc A770 still launches too few work-groups to fill the card (80 for the sliced down kernel, 160 for the multi-token up kernel). On Alchemist this splits them further: **the same values, summed in the same order**, by more groups. `gr_bench` 1.2-1.3x faster, decode +3-5% on an A770, identical outputs. Default on Alchemist only; every other device launches today's kernels. `STRATA_GR_MORE_GROUPS=1` forces it on, `=0` off.

## What changed
`sycl/src/kernels/cuda/fused_gr.dp.cpp` (and one header comment in `sycl/src/kernels/gr_bench.cpp`):
- sliced down kernel: one work-group per (slice, row block), 80 x 6 = 480 groups; each row's partial is computed exactly as before, by a different group.
- `gr_down_reduce`: 8 slice loads in flight per step, then added one at a time, so the sum accumulates slices 0, 1, ..., 79 in the same sequence of float adds as the old loop (`static_assert(GRS_SLICES % 8 == 0)`; `-fp-model=precise`).
- up kernel: templated on the column count, 8 columns per group (320 groups instead of 160) on this path; `UPM_COLS` stays 16 for every other user. Only group 0 writes the per-token `lo` values, which every group computes identically.
- `gr_more_groups(queue)`: once per process, `strata::intel_gpu_gen == Alchemist` by default, env override; independent of `STRATA_GR_LARGE_GRF`.

## How it was checked
Arc A770 16 GB, `xe`, Linux 7.0, oneAPI 2026.1 (JIT), Ryzen 7 3800X, 30 GB RAM; `a0116b27` + #2190 + #2193 with and without this commit, same day.

- `gr_bench`, 1-6 tokens, checksums identical for every token count:

  | build | us (1 / 2 / 3 / 4 / 5 / 6 tokens) |
  |---|---|
  | #2193 | 132.2 / 118.9 / 133.6 / 150.5 / 168.3 / 183.5 |
  | this PR | 116.1 / 94.6 / 105.8 / 122.9 / 136.4 / 148.4 |

- `gr_multi_parity`: 32 cases, 0 failures, with `STRATA_GR_MORE_GROUPS=0` and `=1`. `gr_parity`: only the pre-existing `QFUSE ... not written` failure that `main` has on this card.
- Decode, Coder IQ1_M, 8K-token prompt, 256 tokens, greedy, `--resident-experts --max-context 131072 --kv int8 --kv-resident 20480 --spec 4 --mtp`, `STRATA_VERIFY_STEPPED=1`: #2193 22.12 / 22.35 / 22.55 / 22.65 tok/s -> this PR 23.24 / 23.61 / 23.62 tok/s; output ids identical (`72b762d5b2a3`) in all seven runs.

Not built or run on Battlemage or i915. SYCL-only file.

## Extra Notes
Several other runs of both builds did not finish because of the decode hang reported in #2194; they are left out, not counted against either side.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

本站相关内容

相关页面的快捷入口。