Pull requests / #1247
#1247 prefill: --prefill-help and --prefill-help-frac as the official flags for the idle-stage helper
open · @anon761 · 0 评论 · 在 GitHub 查看
描述
`STRATA_PREFILL_HELP` / `STRATA_PREFILL_HELP_FRAC` already do the work, but env-only is awkward when every other knob has a flag. These two set them before Prefill reads them, so the behavior is identical and repeatable; the envs stay honoured for existing configs.\n\nMeasured on a two-card layer split (2x RTX 3090): about +30% prompt-read time on small prompts (~1.4K tokens), where the second card otherwise sits idle during the first chunk.
站内延伸阅读
链到安装、模型与版本说明,便于 SEO/GEO,非官方 issue 正文。