Issues / #1910
#1910 [Regression] Prefill 35-55% slower on v0.1.42 vs v0.1.41 on dual 2080 Ti (cold long prefix)
open · @wszhoho · 7 コメント · GitHub で見る
BenchmarksMulti-GPUNVIDIA / CUDAModels & quantsLinux
本文
## What do you want to do? **Report a prefill throughput regression between v0.1.41 and v0.1.42** on dual-GPU CUDA hardware, in the hope that it is a fixable regression rather than an intended trade-off. ### Environment - Hardware: 2 × RTX 2080 Ti (11 GB each, 22 GB total VRAM), 62 GB RAM, Linux - Model: IQ3_S (GSQ-RCO, Qwen3.8-Flash-Next family), 256K context - Always measured with a cold 40K-token prompt (27,318 prompt tokens), same prompt across all runs ### Measured prefill (cold 40K prefix) | Build / config | Prefill (tok/s) | |---|---| | v0.1.41 | ~1150–1280 | | v0.1.42, defaults | ~790 | | v0.1.42, `--no-prefill-borrow` | ~565 | | v0.1.42, `STRATA_SPLIT_OWN=1` | ~530 | | v0.1.42, `STRATA_ROUTE_TAIL_SKIP=0` | ~800 (still far below v0.1.41) | So on this hardware v0.1.42 is 35–55% slower than v0.1.41 for a cold long prefix, and none of the exposed knobs (`--no-prefill-borrow`, `STRATA_SPLIT_OWN=1`, `STRATA_ROUTE_TAIL_SKIP=0`) restores v0.1.41 speed. This suggests a lower-level regression in the prefill path (e.g. the BF16 prefill change, chunk scheduling, or the per-card weight/session split) rather than a single settable option. ### Question for the maintainers Was v0.1.42's prefill path changed in a way that could explain this on small-VRAM dual-GPU layer-split setups (22 GB total)? If so, is there an intended opt-in/opt-out equivalent to v0.1.41's behavior that we are missing? (Not a criticism — just flagging the regression so it can be triaged. Happy to provide more measurements or a reproducible script.)
関連リンク
インストール・モデル・リリースへの站内リンク。