Issues / #2377

#2377 [Enhancement] Reduce conversation checkpoint memory with BF16 snapshots

open · @sumomomomomo · 0 コメント · GitHub で見る

本文

FP32 conversation checkpoints consume substantial system RAM, limiting how many conversations and resume points can be retained.

BF16 storage approximately halves checkpoint-state memory. Benchmarks show 12–29% smaller parked images, faster parking, and a small resume overhead, with no observed regression in explicit needle-recall tests. This provides headroom for more cached conversations or denser checkpoints.

Implemented and benchmarked in #2247.

関連リンク

インストール・モデル・リリースへの站内リンク。