Pull requests / #2777
#2777 Start the vision encoder in parallel when it is on another GPU
open · @apatassini · 0 コメント · GitHub で見る
本文
`main()` starts and warms up the encoder (~7 s at 4,096 image tokens) before the engine, so that on the same card it takes its VRAM before the engine sizes its cache. When `vision.cuda_device` is not one of the engine's GPUs that ordering is not needed: the encoder starts on a thread while the engine loads. Cold start 21 → 14 s together with #3. The HTTP server only serves once the engine is up, by which time the encoder is ready. Full notes and measurements: https://github.com/apatassini/Strata/blob/qwen38-16gb/QWEN38.md 🤖 Generated with [Claude Code](https://claude.com/claude-code)
関連リンク
インストール・モデル・リリースへの站内リンク。