Pull requests / #1247
#1247 prefill: --prefill-help and --prefill-help-frac as the official flags for the idle-stage helper
open · @anon761 · 0 comments · View on GitHub
Description
`STRATA_PREFILL_HELP` / `STRATA_PREFILL_HELP_FRAC` already do the work, but env-only is awkward when every other knob has a flag. These two set them before Prefill reads them, so the behavior is identical and repeatable; the envs stay honoured for existing configs.\n\nMeasured on a two-card layer split (2x RTX 3090): about +30% prompt-read time on small prompts (~1.4K tokens), where the second card otherwise sits idle during the first chunk.
Related on strata.com
Editorial links to help you install, pick models, or read release notes — not part of the upstream thread.