#2877 serve: STRATA_PARK_TRUNCATE, park the prefix that fits

说明

# serve: STRATA_PARK_TRUNCATE, park the prefix that fits

Issue: Addresses #1938, #2431

## Summary

The problem this targets is cache utilization when the conversation cache is small relative to the context window (e.g. a 3.5 GiB cache against a 256k-token context): one long-chain snapshot exceeds the whole budget, so slots evict each other entirely and almost no useful prefix survives. Conversation-cache parking and eviction are all-or-nothing: a chain that doesn't fit the byte budget or the RAM floor loses its entire snapshot, and the next request re-prefills from the root-ck (measured 225 s for a 160k-token chain) — the exact failure both #1938 and #2431 describe. Instead of a fixed per-slot preserve limit (#1938's proposal), this uses the conversation's own checkpoint chain as the depth ruler: park the deepest prefix that fits, and shrink already-parked entries to one of their own checkpoints before dropping them. The preserve depth then adapts per conversation to the live byte pressure, instead of one global N. Off unless `STRATA_PARK_TRUNCATE=1` (env-only switch, matching the recent admission-switch convention).

Note: this change was implemented by an AI agent (with the measurements and the repro scenario run live against the engine) — offering it as a reference implementation; rework to fit the codebase's style is welcome.

## What changed

- `include/strata/core/conversation_cache.hpp` (+72/−1)
  - **Park side:** when the outgoing chain fails the byte budget or the `--conversation-cache-min-free-mib` gate, walk its checkpoint chain deepest-first and park the first prefix that passes both gates. The parked copy keeps root + chain + K/V `[0, L)` and mounts live on that checkpoint's running state; the next request mounts the prefix and re-reads only the tail (the reply plus the new turn, which the client re-renders anyway). A truncated copy is parked even below `--conversation-cache-min-tokens` — a root-deep mirror still saves the shared prefix.
  - **Eviction side (`truncate_oldest`):** an already-parked entry asked to make room is shrunk to one of its own checkpoints (K/V pages past the landing, the deeper checkpoints' running states and their ids go; the landing's state moves into live) before it is dropped whole. A landing below `--conversation-cache-min-tokens` is not worth a slot → the entry goes whole. Pinned entries are never truncated; slot-count pressure (`entries_.size() >= slots_`) still evicts whole, since truncation doesn't reduce the count.
  - **Scope:** single-stage only — the layer-split path has no per-checkpoint byte split to walk.
- `src/core/conversation_cache_test.cpp` — 8 new scenarios: truncation keeps the prefix, min-tokens floor evicts whole, switch-off restores old behavior, pinned skip, byte pressure truncates before evicting, slot-count pressure still evicts whole. 4783 checks pass; engine builds clean.

## Extra Notes

Measured on this base (477ac13c). 4-request scenario (grow a 62k chain → two interleaved requests push the byte budget over → return to the first conversation); test config `--conversation-cache-mib 3500`, 2 slots, `min-tokens 4096`, `prompt-cache-every 4096`:

| | patch on | native |
|---|---|---|
| after 3rd request | `parked=2, bytes=3.43G, evictions=0` (first chain truncated to its 32768 landing) | `evictions=1` (first chain dropped whole) |
| return request (62k prompt) | `restored 32768 tokens (live)` → **26.0 s** | full re-prefill → **51.7 s** |
| scenario total | **131.6 s** | **159.3 s** |

Reproduced three times on the previous base with the same delta (~24–26 s on the return leg alone). No flag added on purpose — the env gate keeps the hot path byte-identical when off.