Pull requests / #1264

#1264 verify: the batched K/V append writes a streamed K8V4 state's host copy (#1188, #1169)

closed · @merbanan · 0 评论 · 在 GitHub 查看

描述

#783's batched append of a verify window's cells passed no host copy for the hybrid K8V4 format (int8 and q4_0 pass st.host, and the per-token path passes K8V4's two halves).  With --kv-resident, a block that was not resident when its cells were appended came back from the host copy without them, so the tokens a window wrote were lost: free generation degenerated into repetition (#1188, #1135) and parked conversations lost their decode-written turns (#1169).  The batched path now passes the same host halves as the per-token path.

站内延伸阅读

链到安装、模型与版本说明,便于 SEO/GEO,非官方 issue 正文。