Pull requests / #1394

#1394 serve: evict parked conversations to pass the physical-RAM gate instead of dropping the snapshot

open · @Arthur031221 · 0 评论 · 在 GitHub 查看

Server & APISecurity

描述

## Title
Issue: Resolves #1347

## Summary
With a full conversation cache and little free host RAM, a long conversation wasn't parked even when the parked ones held more RAM than the snapshot needed. The next request had to read its whole prompt again (270k tokens, 96 s, 32 times in the reporter's log).

## What changed
When the plain RAM check refuses, it now calls `malloc_trim(0)` to give freed heap back (glibc keeps freed segments in its heap) and reads free RAM again. If that's still not enough, it evicts the oldest parked conversations one at a time, reading again after each, until the snapshot fits. Pinned prefixes stay, as in `make_room()`. Nothing is evicted if the host reports no figure or the parked ones can't cover it.

## Extra Notes
Builds on the code @alanthinker posted in #1347 (https://github.com/alanthinker/Strata/commit/c62b540). Tested with new cases in `conversation_cache_test` and `conversation_memory_test` (`-DSTRATA_BUILD_CONVERSATION_TESTS=ON`).

I used Claude/Codex to assist with preparing, checking and double-checking this change.

Co-authored-by: Alanthinker Xiang <[email protected]>

站内延伸阅读

链到安装、模型与版本说明,便于 SEO/GEO,非官方 issue 正文。