Pull requests / #2753
#2753 sycl: Q4_K/Q5_K wide GEMV - pass dp4a's ones word as a value IGC cannot fold (it expanded each dp4a into ~24 ops)
open · @aslater3 · 0 comments · View on GitHub
Description
## sycl: Q4_K/Q5_K wide GEMV - pass dp4a's ones word as a value IGC cannot fold (it expanded each dp4a into ~24 ops) The wide (multi-column) Q4_K / Q5_K kernels take the activation sum of the min term as dp4a(0x01010101, u). With the literal constant IGC (oneAPI 2026.1, DG2) does not emit a dp4a: it expands each one into byte movs + add3 (16 movs + 8 add3 per SIMD16 half), ~60% of the 4-column kernel's integer instructions (offline ISA, ocloc acm-g10: 652 -> 320 integer instructions per loop iteration). For 2+ columns the kernels now compute one4 = 0x01010101 + (n_in >> 31), which is 0x01010101 for every n_in >= 0 but a value the compiler cannot prove, and pass it down (WideIQ4XS takes the same argument). One column keeps the literal: there it measured 4-14% faster. The same integers, so the same bits. A770 dense_bench on main, the same output hash in every row: - Q4_K: 1 column -0.2..+5%; 2 columns +6..+12%; 3 columns +10..+16%; 4 columns +12..+18% (e.g. QSA q 145.9 -> 171.7 GB/s, out-proj 139.4 -> 164.9). - Q5_K: 1 column the same (the small k/v and out-proj rows swing +-15% run to run on both builds); 2 columns -2..+3%, k/v -11%; 3 columns +10..+19%; 4 columns +3..+13% (QSA q 162.3 -> 183.1). The engine's verify window runs mostly 4 columns. Engine on a fork tree (with #2207; Coder IQ1_M, 8K code prompt, 3 pairs): GPU wait 32.07 / 31.93 / 31.94 -> 31.85 / 31.63 / 31.59 ms a round, reference tokens. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Related on this site
Quick links to related pages.