Pull requests / #2756

#2756 sycl/A770 prefill: QSA attention 128 queries a launch on Alchemist (bitwise); parity test hashes the FP32 kernel

open · @aslater3 · 0 Kommentare · Auf GitHub

BenchmarksModels & quants

Beschreibung

## sycl/A770 prefill: QSA attention 128 queries a launch on Alchemist (bitwise); parity test hashes the FP32 kernel

pf_attn_batch() / STRATA_PF_ATTN_BATCH: queries per qsa_decode_attn_batch launch in the prompt's QSA attention.
Alchemist default 128 (32 elsewhere, unchanged). Every query's work-groups are its own, so the batch changes no bit;
its scratch (~0.8 MB a query) shares the prompt region with the MoE half's larger scratch.

qsa_prompt_attn_parity: OLDHASH (a hash of the FP32 kernel's output), QPA_OLD_ONLY=1 (only that kernel: the XMX one
refuses an A770, 86,784 B of local memory over its 64 KiB) and QPA_BATCH. A770, 8K context, 2,048 queries:
batch 32 56.0 ms, 64 52.1, 128 50.9, 256 50.3, 512 49.9, the same hash. Engine (Coder IQ1_M, 8K code prompt, 2 pairs,
STRATA_PREFILL_TIMING=1): qsa attn 2269/2270 -> 2077/2078 ms, prefill 712.3/755.5 -> 738.6/763.8 tok/s, reference
tokens in all four.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Mehr auf der Site

Links zu Install, Modellen, Releases.