#1937 serve: parked conversations on the disk, optionally across restarts (--conversation-disk-dir, --conversation-disk-persist)

Description

## What

Opt-in (`--conversation-disk-dir DIR`): a conversation the RAM conversation cache evicts is written to the disk
instead of dropped, and restored when a request continues it. With `--conversation-disk-persist` the conversations also
outlive the engine (second part below). Without the flags nothing changes.

## Why

With `--conversation-cache-mib` an agent's main conversation and its sub-agents can be parked in RAM, but large
conversations do not fit the budget together (here two ~165K-token conversations, ~3.4 GB each in RAM, against an
8 GiB budget): every switch evicted one, and coming back to it read the whole prompt again, ~78 s at 165K tokens on
this box. The live server log showed 13 such evictions and 8 cold re-reads of 64K+ prompts (~520 s) in one session.

## How

- **`ConversationCache::set_on_evict`**: `evict_oldest` hands the image to a callback instead of only dropping it
  (7 lines; `drop_superseded` and `take` do not spill).
- **`ConversationDiskTier`** (`conversation_disk.hpp/.cpp`, host code only):
  - one session file per image of a layer split, through `session_file_write` / `session_file_read` (the same size,
    header and hash checks and the `--session-min-free-mib` preflight); the live state and the deepest checkpoint only
    (the session files' rule); the files' identity is a random number of the run, so only this engine reads them;
  - a background writer thread (CPU nice 10 on Linux): a switch never waits for the disk; a conversation asked for
    again before its write started comes back from RAM, one being written is waited for;
  - `--conversation-disk-ttl-min` (default 30) and `--conversation-disk-mib` (default 32768, oldest first); the tier's
    files are removed at start and at the end;
  - a log line per queue, write, hit and wait (MiB, ms, MB/s) and a totals line after every hit; a request whose
    prompt came back from the disk ends its `prompt N tokens = ...` line with `, reused from the disk in 3.3 s` (the
    read plus the GPU restore), at the end so that the server's and the bench scripts' patterns still match.
- **`generate.cpp`**: the tier beside the RAM cache. `make_ram()` is now the one "room in RAM above the floor" loop
  (wait for the tier's queued writes - they still hold RAM - then evict the RAM cache's oldest), used by parking and
  by a read back. A longer prefix on the disk than in the session, a `--batch` slot or the RAM cache is restored
  through the same validation and restore as a RAM-parked one.
- **`docs/DETAILS.md`**: a section after the conversation cache's settings.

## Measured

2x CMP 50HX (layer split, sm_75), 2x Xeon E5-2670 v2, 64 GB RAM, NVMe ext4, Flash-Next IQ3_XXS, CUDA 13.3, GCC 14;
one RAM slot so that a third conversation pushes one to the disk; greedy, 48 tokens.

Correctness: the follow-up answer of a conversation restored **from the disk** was the same text as the answer the
live conversation gave, for both conversations (42K tokens each); also through the RAM cache.

| 165,557-token conversation | time |
|---|---:|
| cold read | 77.8 s |
| back from the disk, whole request | **6.4 s** |
| of which: read (2,639 MiB in two files) | 3.6 s (761 MB/s) |
| of which: GPU restore | 0.87 s |

The file is smaller than the RAM image (2,639 MiB against 3,364 MB) because only the deepest checkpoint is kept. The
background writes ran at 80-370 MB/s while the next prompt was being read (lower CPU priority); a request waited for
one write once (0.9 s). No file was left after the engine ended.

Tests: new `conversation_disk_test` (55 checks, CPU only, `-DSTRATA_BUILD_CONVERSATION_TESTS=ON`: exact read-back of
a two-stage image, prefix matching, budget, time limit with a fake clock, start-up cleanup, a refused read, a RAM hit
with the writer held, the RAM cache's evictions landing on the disk); clean in 10 ThreadSanitizer runs.
`conversation_cache_test` (4214) and `conversation_file_test` (201) unchanged. Not run on Windows or HIP.

## Second part: conversations that outlive the engine (`--conversation-disk-persist`, opt-in)

### Who it is for, and why

Many people do not keep Strata running all day: they start it for a session of work, stop it to get the GPU and RAM
back, and come back tomorrow. Others restart the server for an update, a settings change or a reboot. Today every
such restart throws away every conversation, and the first request of a long one reads its whole prompt again: an
agent session at 165K tokens costs 77.6 s on this box before the first token, and minutes on smaller ones. The engine
already holds everything needed to continue - the parked conversations are images of exactly that state - so with this
flag they are kept on the disk across restarts, and a long session continues in seconds. The cost is disk space,
bounded by `--conversation-disk-mib` (32 GiB by default), which on a PC with an NVMe is cheap compared with minutes of
GPU time per conversation.

### How

- The files are bound to the **session files' identity** (the model files' fingerprint, the engine version and the
  settings that change the saved bytes) instead of a random number of the run, so only a compatible engine reads them;
  everything else (another model, engine version or settings, a damaged index, an interrupted write) is removed at start.
- A small **index file** per conversation (its token ids and write time, published by rename after its images) lets
  the next start take them in without reading gigabytes.
- **No time limit** in this mode: the budget alone decides, least recently written first.
- **On QUIT** (the server's stop, Ctrl+C, SIGTERM) the engine parks the current conversation, moves every parked one
  to the disk and waits for the writes before it ends; the server waits up to 300 s for that (20 s otherwise). The
  engine ignores SIGINT in this mode: it is in the server's process group, so a terminal's Ctrl+C used to end it
  before any save; now the server's QUIT ends it after the save, and a second Ctrl+C still ends it at once.

### Measured (same box)

The 165,557-token conversation current at the stop, two 42K-token ones parked:

| | |
|---|---:|
| save at the stop (all three) | **6.8 s** (the 165K one: 2,639 MiB written in 2.9 s, 956 MB/s) |
| next start | took in 3 conversations (4,311 MiB), refused none |
| the 165K conversation's first request after the restart | **6.7 s** instead of **77.6 s** |
| answers after the restart | the same greedy text as before it, for all three |

`conversation_disk_test` now also covers the restart (files kept, taken in, read back exactly, another identity's files
and a damaged index refused); `serve.test_server` 291 tests pass.

## Related: #1959

Measuring this tier on real agent sessions (the pi coding agent) showed a different loss: after the client rewrote its
history a few thousand tokens back, a 210K-token conversation that was still held resumed from the system prompt and
was read again (100 s), because least recent use keeps all checkpoints near the end. That is not about parked
conversations, so it is its own PR: #1959 (`--prompt-cache-spread`, checkpoints kept spread back from the end).

## How to try

Add to the config's `args`, next to the conversation cache:

```
"--conversation-disk-dir", "/path/to/cache-folder"
```

and, to keep the conversations across restarts, also `"--conversation-disk-persist"`.

and look for `strata serve: conversation disk:` lines in the engine log.

🤖 Generated with [Claude Code](https://claude.com/claude-code)