#2325 --batch: keep system-prompt roots apart from the conversations (--prompt-cache-roots, opt-in)

Description

## Why

We serve an agent client whose every chat opens with the same ~79K-token system prompt and tool list, on `parallel: 4` with `--batch-mtp` and `--conversation-cache-mib 8192`. Its chats overlap with other traffic all day, and every new chat read the whole opening again (20-35 s) although the engine had read the same tokens minutes before.

The cause is in `--batch`. The root checkpoint (the end of the system prompt, PR #65) lives in the main session's chain, but:

- a slot keeps only its turn checkpoint (the deepest one) at admission;
- a request that continues from a slot clears the main chain and takes over that one checkpoint;
- the next park then drops the older parked copies that still held the root (#342's `drop_superseded`).

After a few turns of any chat nothing holds the root any more. Solo traffic does not show it: a request that runs alone stays on the main session and its chain stays whole.

## What it does

`--prompt-cache-roots N` (default 0 = off, every path unchanged) keeps a copy of the last N roots the engine made, apart from any conversation (one per distinct system prompt, ~118 MB of RAM each; least recently made or used leaves first). A root's state depends on its tokens alone, so:

- **main session:** at a request's start, and after a slot or a parked conversation was restored, a kept root goes back into the chain (at its front, where retention pins it) when the session's cells hold its tokens and the prompt starts with them;
- **idle slot:** a kept root whose tokens an idle slot holds is a source like the slot's own checkpoints: the slot's K/V up to the root is copied back and the root's state restored;
- **#879** (non-finite logits) drops the kept roots with everything else.

Because the main chain keeps its root again, parked conversations keep it too, so a new chat also mounts through a parked one.

## Tests

`conv_cache_test` covers the store (kept, refreshed, the least recently used one leaves, other images or control-vector mode do not match, clear).

**Real engine:** v0.1.43 on one RTX PRO 6000 (Blackwell), Qwen3.8-Flash-Next IQ3_S, `parallel: 4`, `--batch-mtp`, `--conversation-cache-mib 8192`. A 42K-token system prompt; chat A three turns, a request with another system prompt, chat B two turns, chat C. Greedy, the same requests with the flag off and on:

| | flag off | flag on |
|---|---|---|
| sequential (no overlap) | new chats reuse 42,146 tokens | the same |
| two other requests always decoding: new chat B (`--prompt-cache-roots 2`) | **0 reused, 11.6 s** | **42,146 reused, 0.8 s** |
| parking off (`--conversation-cache-mib 0`), the root only in an idle slot: new chat B (`--prompt-cache-roots 8`) | **0 reused, 7.9 s** | **42,146 reused, 0.8 s** |

Answers were identical between the arms except one 600-token answer decoded beside another request; that one also differs between two runs with the flag off (batched greedy decode is not run-to-run identical).

N matters: in the last scenario with N = 2, a third system prompt pushed the 42K root out before chat B came, so we run 8.

**In use:** v0.1.43 plus this change has run with `--prompt-cache-roots 8` on two RTX PRO 6000s for about 8 hours of mixed agent, chat and tool traffic (about 2,800 requests, no errors); new chats of that client now reuse their shared opening.

**Builds:** the measurements above are from this change on v0.1.43. Rebased on current `main`, it builds for CUDA (sm_120) and CTest passes (`conv_cache_test` included) except `ple_parity`, which needs a Q2_0 model file I do not have. I could not build HIP or SYCL (no toolchain here). The change is host-only C++ in `src/program/generate.cpp` and `conv_cache.hpp`; the SYCL `generate.cpp` includes the header but does not use the new class (the header also compiles alone with g++ `-Wall -Wextra`).

**Default path:** with the flag at 0 the store is empty and every new branch is skipped: `seed_root`, the slot lookup and `keep` all test `enabled()` or `prompt_cache_roots` first, so no checkpoint, copy or log line differs.

Related: #1529 keeps the root in the disk tier's spilled files. This change is the RAM side under `--batch` and does not touch the disk tier.

Happy to change the shape (a default other than 0, or keeping the roots inside the conversation cache's budget).

🤖 Generated with [Claude Code](https://claude.com/claude-code)