Pull requests / #2753

#2753 sycl: Q4_K/Q5_K wide GEMV - pass dp4a's ones word as a value IGC cannot fold (it expanded each dp4a into ~24 ops)

open · @aslater3 · 0 commentaires · Sur GitHub

Models & quants

Description

## sycl: Q4_K/Q5_K wide GEMV - pass dp4a's ones word as a value IGC cannot fold (it expanded each dp4a into ~24 ops)

The wide (multi-column) Q4_K / Q5_K kernels take the activation sum of the min term as dp4a(0x01010101, u). With the
literal constant IGC (oneAPI 2026.1, DG2) does not emit a dp4a: it expands each one into byte movs + add3 (16 movs +
8 add3 per SIMD16 half), ~60% of the 4-column kernel's integer instructions (offline ISA, ocloc acm-g10: 652 -> 320
integer instructions per loop iteration). For 2+ columns the kernels now compute one4 = 0x01010101 + (n_in >> 31),
which is 0x01010101 for every n_in >= 0 but a value the compiler cannot prove, and pass it down (WideIQ4XS takes the
same argument). One column keeps the literal: there it measured 4-14% faster. The same integers, so the same bits.

A770 dense_bench on main, the same output hash in every row:
- Q4_K: 1 column -0.2..+5%; 2 columns +6..+12%; 3 columns +10..+16%; 4 columns +12..+18% (e.g. QSA q 145.9 -> 171.7
  GB/s, out-proj 139.4 -> 164.9).
- Q5_K: 1 column the same (the small k/v and out-proj rows swing +-15% run to run on both builds); 2 columns -2..+3%,
  k/v -11%; 3 columns +10..+19%; 4 columns +3..+13% (QSA q 162.3 -> 183.1).
The engine's verify window runs mostly 4 columns. Engine on a fork tree (with #2207; Coder IQ1_M, 8K code prompt,
3 pairs): GPU wait 32.07 / 31.93 / 31.94 -> 31.85 / 31.63 / 31.59 ms a round, reference tokens.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Sur le site

Liens install, modèles, releases.