Pull requests / #225
#225 perf(prefill): honor explicit routed-only staging for large chunks
closed · @rluisr · 0 commentaires · Sur GitHub
Description
## Summary Split the staging-policy change out of #139 as requested in https://github.com/Niko1221/Strata/pull/139#issuecomment-5898192943. An explicit `STRATA_PREFILL_RING=8` now opts into the existing routed-only staging path even for chunks of 2048 tokens or more. Previously the value was clamped to 16, so it kept all-expert streaming enabled instead of honoring that opt-in. - One production-code line; no GEMM or architecture changes. - Defaults remain 384 slots for mostly pinned experts and 96 otherwise at large chunk sizes. - Small chunks still use 8 slots. - Other explicit values retain the existing 16–512 clamp. This is an opt-in tradeoff, not a new default: routed-only staging can reduce host-staging pressure but forfeits overlap from all-expert prefetch. The original combined V100 tuning evidence is retained as historical evidence in #139; it does not isolate this line's contribution. ## Validation Compiled and executed a host-side assertion harness using the exact `ring_slots` function from this branch. Verified: - Default pinned/unpinned behavior. - The 2047/2048-token boundary. - Explicit `8` for 1, 2047, 2048, 4096, and 8192 tokens. - Unchanged clamping for explicit 0, 7, 9, 16, 96, 512, and 513. No isolated end-to-end staging speedup is claimed.
Sur le site
Liens install, modèles, releases.