Pull requests / #2019

#2019 prefill: q's split + norm + rope as one kernel (STRATA_QPREP=1); gate_attn in the prompt attention's epilogue (STRATA_ATTN_GATE=1)

open · @libratechw · 0 コメント · GitHub で見る

AMD / HIPNVIDIA / CUDA

本文

## Title
Issue: none


## Summary
Two opt-in fusions on the full-attention layers' prompt path, both bitwise equal to the kernels they replace. The gains are
small (q_prep inside the noise of 3 rounds; the gated epilogue -0.9% of a 32K request, one trace), so they stay opt-in. Say if
you would rather not carry them.

## What changed
`prefill.cpp`, `kernels.cu/.hpp` (q_prep), `qsa_prompt_attn.cu/.hpp` (GATED epilogue, HIP STRATA_PA_FAST kernel only), two new
tests: `hip_prefill_q_prep` (bitwise vs split_q + rms_rows + rope, 1..4,089 tokens, positions 0 / 4,089 / 100,000: 0 differ) and
`hip_prompt_attn_wmma_gated` (FP16 bits vs attention + gate_attn: 0 differ). Both pass on gfx1151.
Measured: gate_attn's 152 ms gone from a 32K prompt, request 22.97 -> 22.77 s. Same text in every round.
Not built: CUDA, SYCL.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

関連リンク

インストール・モデル・リリースへの站内リンク。