#6440 [Engine] Release duplicate dense-loader weights on resident Metal
open · @Maryarty · 0 comentários · No GitHub
Descrição
## Problem On my M1 Max with 64 GiB, the single-window resident Metal path constructs the common dense-loader copies, then executes against the weights mapped from the source GGUF. With Qwen3.8-Flash-Next GSQ-RCO Q2_0 and PLE on Strata main `3142b66`, the extra copies can push a resident prefill over the local memory guard even after the engine reaches READY. ## Proposed scope I'd like to propose two small opt-in changes for the single-window resident text path. The first releases the common pack arena after Metal has selected resident mode and before it wraps the GGUF. The second releases the separately owned NativeDense projections while preserving the PLE key and leaving its optional host conversion scratch alone. Each owner has its own flag. The selection honors `STRATA_METAL_KEEP_HOST_WEIGHTS=1` and is limited to the text path without batch, vision, prefill head aliasing or DFlash2. This uses the allocation owners and the existing Metal residency-set mechanism. ## Measurements The upstream-based prototype builds with Metal and passes `runtime_hello`. Each 2K run used the same M1 Max, Q2_0 pack, 2,304 context cap, 256-token prefill chunk and MTP width 2. | Flags | Process footprint at engine READY | Result | | --- | ---: | --- | | Off | 5.909 GB | Host guard stopped during the first request | | Pack arena | 4.424 GB | Host guard stopped during the first request | | NativeDense | 4.472 GB | Host guard stopped during the first request | | Both | 2.992 GB | 128 output IDs completed, minimum 18% system free | All four recorded zero new system swapouts. In a separate 16-ID decode control with the draft model unloaded, both-off and both-on produced bitwise-identical first logits and all 128 greedy output IDs matched. The 2K series produced one complete arm, so a full parity pair with MTP is still open. Would this opt-in owner-by-owner scope fit Strata? I can submit the pack-arena part first and follow it with NativeDense.
No site
Links install, modelos, releases.