改动 / #2752

#2752 sycl: Q6_K GEMV - subtract 32 from the 6-bit values with three integer ops instead of a saturating byte subtract

open · @aslater3 · 0 评论 · 去 GitHub 看

BenchmarksModels & quants

说明

## sycl: Q6_K GEMV - subtract 32 from the 6-bit values with three integer ops instead of a saturating byte subtract

The Q6_K dot paths took 32 from each 6-bit weight with dpct::vectorized_binary<char4>(v, 0x20202020, sub_sat()), which
unpacks and saturates byte by byte. The bytes are 0..63, so the result never leaves int8 range: q6_sub32(v) =
((v | 0x80808080) - 0x20202020) ^ 0x80808080 gives the same bytes (bit 7 set keeps one byte's subtract from borrowing
from the next; flipping it back restores the sign). Five sites. Hardware counters showed these kernels integer-ALU
bound, not memory bound.

A770 dense_bench, same output hashes: Q6_K GDN qkv 4 columns 239 -> 289 GB/s, 1 column 321 -> 366; QSA q 4 columns
242 -> 293; k/v 72 -> 94. Engine (Coder IQ1_M, 8K code prompt, 3 pairs): GPU wait 33.4-33.8 -> 32.0-32.2 ms a round,
decode 39.1 / 39.2 -> 39.6 / 39.4 tok/s, the same tokens. Measured on a tree that also has #2207 (1-column MMVQ in
large GRF). All devices: the arithmetic is the same integers.

Re-measured on main (A770 dense_bench, this branch vs main, the same hash in every row): Q6_K GDN qkv 4 columns
94.2 -> 159.6 GB/s, 3 columns 96.6 -> 197.1, 1 column 288.9 -> 310.5; QSA q 4 columns 92.2 -> 164.9; GDN z 4 columns
87.8 -> 138.9; out-proj 4 columns 119.5 -> 138.4; k/v 1 column 57.7 -> 88.9. Larger than on the fork tree above, which
also has #2207.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

本站相关内容

相关页面的快捷入口。