Pull requests / #2761
#2761 what only the decode needs leaves VRAM while a prompt runs, and the prestage ring takes the loan (WIP - porting mw00/project-maya#223)
open · @gopinath87607 · 0 コメント · GitHub で見る
本文
Placeholder - nothing to review yet. This PR claims the line of work; the port lands here in later commits, gated and off by default. While a prompt runs, memory only the decode reads leaves VRAM and is back before the next token: the DFlash drafter's weights and block buffers (pinned host copies once, a DMA back, its K/V ring and pinned staging kept), and in an MTP config the NextN block's expert slots - already inside the pool's lendable tail, so a prompt's carve takes them first. Those bytes become prestage ring: on a split with the ring's cap unset, only the pool's lend bounds it (measured in mw00/project-maya#223: the cap of 160 experts became 439 and 303 on the two parts that sat at the constant). The shape here follows Strata's code, not Maya's. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
関連リンク
インストール・モデル・リリースへの站内リンク。