Pull requests / #1264
#1264 verify: the batched K/V append writes a streamed K8V4 state's host copy (#1188, #1169)
closed · @merbanan · 0 comentarios · En GitHub
Descripción
#783's batched append of a verify window's cells passed no host copy for the hybrid K8V4 format (int8 and q4_0 pass st.host, and the per-token path passes K8V4's two halves). With --kv-resident, a block that was not resident when its cells were appended came back from the host copy without them, so the tokens a window wrote were lost: free generation degenerated into repetition (#1188, #1135) and parked conversations lost their decode-written turns (#1169). The batched path now passes the same host halves as the per-token path.
En el sitio
Enlaces a install, modelos, releases.