Pull requests / #2312

#2312 fix(calibrate): keep configured vision resident during tuning

open · @Tsury · 0 コメント · GitHub で見る

Setup & installAMD / HIPNVIDIA / CUDAModels & quantsLinux

本文

Calibration sized the automatic expert cache without the configured vision encoder, although normal serving loads vision first. On the same Lenovo configuration, engine-only startup allocated 8343 expert slots/16202 MiB; vision-first serving allocated 7615/14786 MiB.

Keep the configured `Vision` resident throughout the tuner’s engine restarts. Use serving’s path normalization and vision environment, then shut down the encoder and close its log on success or failure. Text-only configurations and caller-supplied engine factories retain their existing behavior. The tuner’s prompts, sweeps, thresholds, sampling, and selected settings logic are unchanged.

Validation: 23 focused calibration/setup tests pass. Five new failure cases reproduced the missing encoder before the fix. Coverage checks GPU/CPU vision lifetime, restarts, path/environment handling, encoder/engine/measurement failures, log cleanup, and text/custom-factory behavior. No native C++/CUDA/HIP/SYCL build inputs changed.

The allocation observations used Linux, RTX 5090 Laptop 24 GiB, Core Ultra 9 275HX, SC117 Swift 1.5 IQ3_S, context262144, INT8/resident32768, workers7, PCIe0.55, and VRAM reserve1600 MiB. Both diagnostics used the same native/vision executables and generated no tokens. The vision-resident supported stock tuner also completed; any selected-settings performance observation is separate from this initialization repair. Other GPU platforms were not measured here.

Fixes #2301.

関連リンク

インストール・モデル・リリースへの站内リンク。