#2264 Conversation cache re-parks the same conversation every turn: 2251.9 GB of host copies at 4.7 % reuse, and the K/V retention that would make the write incremental can never apply

Beschreibung

### Summary

Once a conversation has been parked, the engine parks it again on every following
request. Each turn writes the full snapshot to host RAM, `drop_superseded()` deletes the
copy written one turn earlier, and the K/V retention that would reduce the write to the new
suffix never engages. Measured over **1,418 successful parks** in one server log:

| | |
|---|---|
| successful parks | 1,418 |
| host copy volume | **2,251.9 GB** (1.59 GB each) |
| volume ever read back | 105.8 GB (**4.7 %**) |
| events with zero reuse | **80 %** |
| superseded copies dropped | 425 |
| restores | 355 |
| park refusals, free RAM measured | 325 |
| park refusals, no telemetry | 1,441 |
| copy time on the request path | **1,548 s (25.8 min)**, median 234 ms, worst 2,075 ms |

Every figure is counted over the same window of one server log (`strata-iq3_xxs.log`, up to
and including the park at line 80035; the server restarted right after, which is why the whole
log holds 19 more parks). The counters, so the numbers can be re-derived rather than taken on
trust:

```
grep -cE 'conversation cache: parked [0-9]+ tokens'                  -> 1418 in window (1437 whole log)
grep -cE 'conversation cache: restored [0-9]+ tokens'                ->  355
grep -cE 'conversation cache: dropped [0-9]+ superseded'            ->  425 in window (435 whole log)
grep -cE 'skip parking \(physical RAM admission; need [0-9]+ MiB plus [0-9]+ MiB floor, [0-9]+ MiB available' -> 325
```

The refusal classes are counted separately, because they say different things: 1,766 physical-RAM
admission in the window, 185 because the snapshot exceeded the cache budget, 6 and 3 after capture
for the RAM floor. The 1,989 refusals over the whole log are therefore *not* 325 - the 325 are the
subset that quotes measured free RAM.

Three consecutive snapshots in one turn differed by **6,152 bytes out of 2.4856 GB**
(0.0002 %). A growth gate cuts the per-benchmark volume to 3.26 GB (-66.7 %) and removes the
superseded class entirely; the measurement is below and a cleaned-up version of the same
gate is in #2265.

Note the cost class: this is host-RAM churn plus PCIe/memcpy time, not SSD I/O.
`--conversation-cache-mib` is documented as a *RAM* budget ("RAM budget for parked
conversations"), and each `ConversationBuffer` is "a list of 16 MiB segments, each segment its
own allocation", so no filesystem is in the path. That makes it a plausible contributor to
swap pressure.

### Environment

| | |
|---|---|
| GPU | NVIDIA GeForce RTX 5090 Laptop GPU, 24 GiB, PCIe Gen5 **x8**, probe 28.7 GB/s |
| CPU | Intel Core Ultra 9 275HX, 8 P + 16 E cores, 24 logical |
| RAM | 64 GiB physical, **WSL limit 54 GB**, expert arena 39.97 GiB in 42 GiB hugepages |
| OS | Ubuntu 24.04 under WSL2 |
| Model | Qwen3.8-Flash-Next **IQ3_XXS**, native pack, 10,320 expert slots / 16.78 GiB cache |
| Engine | `a0116b27` (v0.1.43.1, current `origin/main`), nvcc 13.4.92, arch 120 |

Config, identical in every arm:

```
--conversation-cache-mib 4096 --conversation-cache-slots 4
--conversation-cache-min-free-mib 1536 --prompt-cache 32 --prompt-cache-every 8192
--pool-workers 12 --pool-tasks 192 --kv-grow --kv k8v4 --vram-reserve-mib 300
--max-context 262144 --spec 4 --spec-min-p 0.5 --prefill auto:16384
env: STRATA_PF_FUSED=1, STRATA_ADAPT_OVERLAP=0
```

Single long agentic conversation, 100k-185k tokens. **No switching between conversations is
required** - this reproduces with one conversation only.

### Root cause

`src/program/generate.cpp`, the park decision (line 10445 at `a0116b27`):

```cpp
if ((!from_live || incoming || slot_source >= 0) && !pin_sibling && !park_current(incoming ? incoming->bytes() : 0)) {
```

The trigger is `!from_live`, not `incoming`. Parking does set `incoming` for the next
request, but a probe compiled into the binary shows it is **0 on every turn** and that
`!from_live` is the sub-condition that fires:

```
strata serve: [park-probe] from_live=0 incoming=0 slot_source=-1 pin_sibling=0
                          park_resumed=0 keep_resumed=0 resume=158108 live=158312
```

That measurement is the finding; what follows is my reading of it, offered as the likely
mechanism rather than as something I proved. `from_live` is set only where `live_ok &&
starts_with(live, live_imgs)` holds (line 10378), and the checkpoint loop right after it
resets it:

```cpp
if (live_ok && starts_with(live, live_imgs)) { resume = (int64_t) live.size(); from_live = true; }
for (const ConvCheckpoint& c : checks)
    if ((int64_t) c.ids.size() > resume && starts_with(c.ids, c.imgs)) {
        resume = (int64_t) c.ids.size();
        from_live = false;
    }
```

So once the live state is gone - and a park leaves the session "in between" until the request
finishes - the next request arrives with `from_live=0` and parks again. **The park triggers
the next park.** Which of the two paths above clears it is worth a maintainer's look; I only
know the condition holds every turn.

Two findings compound it:

1. **The K/V retention can never apply.** The documented rule is that the estimate is
   "the snapshot's estimated size, less the K/V it already retains":

   ```cpp
   const size_t retained   = reuse.bytes() + stage_retained;
   const size_t additional = estimate > retained ? estimate - retained : 0;
   ```

   but in this path `retained` is ~1 MiB against a 3,113 MiB estimate, and the log confirms
   it: `need 3112 MiB` for a `snapshot_bytes=3264196896` (3,113 MiB) snapshot, with
   `reused_kv_bytes=0` on every event. `reuse` comes from `conversations.take_reuse()`, but
   `reuse_` is only ever filled by `conversations.retain(...)` inside the *restore* block
   (`generate.cpp:10542`). In the same window the cache parks 1,418 times and restores 355, so on
   most turns no restore happens, `retain()` is never reached, and `take_reuse()` correctly has
   nothing to hand back. The empty credit is a consequence of the park/restore ratio, not of
   the order of two statements inside the park path.

2. **The memory guard refuses exactly when it would help.** Parking needs host RAM to
   serialise into, so under pressure it declines:

   ```
   conversation cache: skip parking (physical RAM admission; need 3112 MiB
                       plus 1536 MiB floor, 3885 MiB available)      [325x]
   ```

   So the case parking exists for - RAM too tight - is the case where it cannot run, and the
   case where it does run produces a snapshot that is discarded next turn.

   One caveat on that 325: it is the subset of refusals that **quotes measured free RAM**. The
   remaining 1,441 refusals in the same window print `telemetry unavailable` instead - the same
   refusal, but with no measurement behind it, so for those the log does not record whether the
   snapshot would have fitted. The 325 are what the log can prove; the refusal rate itself is
   1,960 in this window.

### Reproduction

```
conversation cache: parked 100821 tokens in ...; parked=1 bytes=2485686... reused_kv_bytes=0
conversation cache: parked 100820 tokens in ...; parked=1 bytes=2485648... reused_kv_bytes=0
conversation cache: dropped 1 superseded copy of this conversation; parked=0
conversation cache: parked 100819 tokens in ...; parked=1 bytes=2485639... reused_kv_bytes=0
```

Parked tokens: 100821 -> 100820 -> 100819. Size spread across the three: **6,152 bytes**.
Reuse: 0, 0, 0.

### The gate

A conversation that advanced by a few hundred tokens since its last park is not worth a 3 GB
copy that the next turn deletes: the stored entry still holds this conversation's prefix, and
a conversation only appends, so it stays usable for the reuse it was made for.

PR (linked below) turns that into `--conversation-cache-park-gain N`, **default 0 = park every
turn, the current behaviour**, so nothing changes until it is asked for. What I measured on my
rig is the same gate as an env switch (`STRATA_PARK_RESUMED`) with the threshold hard-coded at
32,768 tokens; the pull request makes the threshold a flag and leaves the default off.

### Measurement protocol

Single patched binary, switched by env only - no rebuild between arms. Same config, same
prompt (158,113 tokens), 5 generations of 256 tokens, medians from the server's own log line.
Park figures counted over the log line range of each arm.

| | Arm A (upstream) | Arm C (gate on) |
|---|---|---|
| park events | 3 | **1** |
| host copy volume | 9.79 GB | **3.26 GB (-66.7 %)** |
| superseded copies dropped | 2 | **0** |
| refused park attempts | 1 | **0** |
| decode median | 120.5 tok/s | 116.4 tok/s |
| decode min / max | 92.2 / 124.1 | 93.8 / 123.2 |
| expert cache hit rate | 91.5 % | 91.3 % |

The remaining 3.26 GB is the **first** park of the conversation, which is a real state change
and should stay.

**The decode difference is -3.4 % and is not significant**: it is inside the run-to-run spread
(92.2-124.1 tok/s in the same arm). The expert-cache hit rate is unchanged (91.5 % vs 91.3 %),
so keeping the K/V resident in VRAM does not cost cache hits here. I do not claim a decode
win; the claim is the removed copy volume and the eliminated superseded class.

### Quality verification

Two tests, both on the gated build:

1. **Artifact check** - letter counting, 17x23, capitals, Python function: 4/4 correct, zero
   repetition.
2. **Conversation integrity** - five facts stated in turn 1 (PIN 7413,
   "vierzehntausendneunhundert", Rostock, Axolotl, `y = 3x + 7`), then four filler turns to
   grow the context, then all five queried again: **5/5 recalled correctly**, no repetition,
   K/V intact. This is the test that matters for a change on the snapshot path, because a
   broken restore shows up as a lost early fact, not as nonsense in a single answer.

### Suggested upstream fix

1. **Rate-limit parking by growth.** The gate in the linked pull request. A conversation that
   advanced a few hundred tokens since its last park does not need a fresh 3 GB copy.
2. **Do not spend a park on a conversation you are not going to restore.** Counting the two line
   types over the same window gives **1,418 successful parks against 355 restores** - four parks
   per restore. (Both numbers are counted from the log lines, not printed by the engine: its own
   per-event lines carry the counters, `parked=%zu` on the park line and
   `restored %lld tokens` on the restore line.)

   This is *why* `reused_kv_bytes` is always 0, and it is not an ordering problem: `conversations.retain(...)` (which fills `reuse_`) sits inside the restore
   block at `generate.cpp:10542`, guarded by `incoming` having been successfully taken. Since
   the benchmark conversation is never restored, `retain()` never runs, `reuse_` stays empty,
   and `take_reuse()` correctly returns nothing - there is nothing to retain. Reordering
   `drop_superseded()` and `take_reuse()` would change nothing: `take_reuse()` reads what a
   *previous* restore retained, not what the cache still holds. The empty `retained` is a
   symptom of the park/restore ratio, not a sequencing defect.
3. **Make the live-state invalidation explicit.** Whatever marks the live state unusable after
   a park should also record that this conversation was just parked, so the next turn does not
   re-enter the same branch. A flag like "parked this conversation, it is the one being
   continued" is cheap and directly breaks the loop.
4. **Report both halves in the log.** The park line prints `snapshot_bytes` but not the bytes
   this call wrote, and the refusal line prints `need` without `retained`. With the current
   output there is no way to tell that the documented K/V credit never applies.
5. **Consider the default.** The pull request ships the gate with default 0, which changes
   nothing. The measurement above is for one rig; whether the gate should be on by default is
   your call, and it is the only question the numbers there really open.

### Impact

On a long single-conversation workload this is the largest steady-state cost I measured
out