#5504 [Engine] Credit clean active_file cache in the conversation memory guard
closed · @giphoo · 0 コメント · GitHub で見る
本文
## Problem and result The conversation memory guard refused session reads on an mmap-heavy engine because `reclaimable_cache()` only credited `inactive_file`, giving 0 reclaimable bytes even when 17+ GiB of clean file cache sat on the active LRU list. After this change, both `inactive_file` and `active_file` are credited (less dirty and writeback, capped at the cgroup's current usage). ## Related discussion none ## Approach The current comment says `active_file (it holds mlocked pages)` - but on cgroup v2, mlocked pages are counted in `unevictable`, not `active_file`. Clean `active_file` pages are regular file cache on the active LRU list and are reclaimable by the kernel. An mmap-heavy engine (model weights read through mappings) naturally keeps its file cache on the active list. Measured on a production system: ``` inactive_file: 0 active_file: 17.5 GiB file_dirty: 0 unevictable: 0 ``` The guard read 0 credit and refused a 50 MiB session file read with 28 GiB actually free. This PR credits `inactive_file + active_file`, still less dirty and writeback, still capped at the cgroup's charged usage, and still does not credit shmem (the KV pool's pinned complements). ## Validation **Passed (reference implementation, Linux, cgroup v2, mmap-heavy workload):** - Session read admission succeeds when active_file holds the file cache - 51-request conversation rotation with 0 OOM after the change **Failed:** none observed **Not run:** - cgroup v1 (`total_active_file` path is added but untested) - Windows (no cgroup memory.stat) - CUDA, SYCL, Metal (host-side only) ## Limits - Crediting `active_file` assumes the kernel can reclaim those pages under pressure. On a system with genuine memory shortage, reclaiming active file cache causes I/O to re-read the pages later, trading latency for admission. - cgroup v1 path added but not tested.
関連リンク
インストール・モデル・リリースへのサイト内リンク。