Pull requests / #1394

#1394 serve: evict parked conversations to pass the physical-RAM gate instead of dropping the snapshot

open · @Arthur031221 · 0 comments · View on GitHub

Server & APISecurity

Description

## Title
Issue: Resolves #1347

## Summary
With a full conversation cache and little free host RAM, a long conversation wasn't parked even when the parked ones held more RAM than the snapshot needed. The next request had to read its whole prompt again (270k tokens, 96 s, 32 times in the reporter's log).

## What changed
When the plain RAM check refuses, it now calls `malloc_trim(0)` to give freed heap back (glibc keeps freed segments in its heap) and reads free RAM again. If that's still not enough, it evicts the oldest parked conversations one at a time, reading again after each, until the snapshot fits. Pinned prefixes stay, as in `make_room()`. Nothing is evicted if the host reports no figure or the parked ones can't cover it.

## Extra Notes
Builds on the code @alanthinker posted in #1347 (https://github.com/alanthinker/Strata/commit/c62b540). Tested with new cases in `conversation_cache_test` and `conversation_memory_test` (`-DSTRATA_BUILD_CONVERSATION_TESTS=ON`).

I used Claude/Codex to assist with preparing, checking and double-checking this change.

Co-authored-by: Alanthinker Xiang <[email protected]>

Related on strata.com

Editorial links to help you install, pick models, or read release notes — not part of the upstream thread.