Pull requests / #2761

#2761 what only the decode needs leaves VRAM while a prompt runs, and the prestage ring takes the loan (WIP - porting mw00/project-maya#223)

open · @gopinath87607 · 0 comentarios · En GitHub

Descripción

Placeholder - nothing to review yet. This PR claims the line of work; the port lands here in later commits, gated and off by default.

While a prompt runs, memory only the decode reads leaves VRAM and is back before the next token: the DFlash drafter's weights and block buffers (pinned host copies once, a DMA back, its K/V ring and pinned staging kept), and in an MTP config the NextN block's expert slots - already inside the pool's lendable tail, so a prompt's carve takes them first. Those bytes become prestage ring: on a split with the ring's cap unset, only the pool's lend bounds it (measured in mw00/project-maya#223: the cap of 160 experts became 439 and 303 on the two parts that sat at the constant).

The shape here follows Strata's code, not Maya's.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

En el sitio

Enlaces a install, modelos, releases.