Pull requests / #225

#225 perf(prefill): honor explicit routed-only staging for large chunks

closed · @rluisr · 0 comentarios · En GitHub

Descripción

## Summary

Split the staging-policy change out of #139 as requested in https://github.com/Niko1221/Strata/pull/139#issuecomment-5898192943.

An explicit `STRATA_PREFILL_RING=8` now opts into the existing routed-only staging path even for chunks of 2048 tokens or more. Previously the value was clamped to 16, so it kept all-expert streaming enabled instead of honoring that opt-in.

- One production-code line; no GEMM or architecture changes.
- Defaults remain 384 slots for mostly pinned experts and 96 otherwise at large chunk sizes.
- Small chunks still use 8 slots.
- Other explicit values retain the existing 16–512 clamp.

This is an opt-in tradeoff, not a new default: routed-only staging can reduce host-staging pressure but forfeits overlap from all-expert prefetch. The original combined V100 tuning evidence is retained as historical evidence in #139; it does not isolate this line's contribution.

## Validation

Compiled and executed a host-side assertion harness using the exact `ring_slots` function from this branch. Verified:

- Default pinned/unpinned behavior.
- The 2047/2048-token boundary.
- Explicit `8` for 1, 2047, 2048, 4096, and 8192 tokens.
- Unchanged clamping for explicit 0, 7, 9, 16, 96, 512, and 513.

No isolated end-to-end staging speedup is claimed.

En el sitio

Enlaces a install, modelos, releases.