#2510 serve: durable on-disk conversation tier, per-stage segments, prefix restore ladder and Windows chunked large pages

Descripción

## Title
serve: a durable on-disk conversation tier with a warm→cold cascade that degrades instead of deleting, per-stage segments, a prefix restore ladder, Windows chunked large pages, and a parking target (ram|disk|auto) that spills a rejected park

Issue: Resolves #2597

## Summary

Four bodies of work in one series; all of it opt-in and off by default.

The **first eight commits** (already on this PR) add a durable disk tier for the conversation cache, a per-stage
segment tier, a prefix restore ladder and Windows chunked large pages. With every flag absent the tree is
byte-for-byte upstream, and the series applies clean in order on the base.

The **parking work** adds a parking target (`--conversation-cache-park ram|disk|auto`) and the rule that closes the
hole the disk tier was built for: **when the physical-RAM gate rejects a park, the capture is no longer discarded —
it spills to disk under a transient floor.** The order is **RAM → disk → discard**, and a discard is now *counted*
and printed (`discards=N`). The default stays `ram`, so with no flag the behaviour is the previous one plus that
fallback.

The **cascade** (new) gives the disk tier a level above it. The conversation cache now runs **RAM → warm M.2 →
cold disk**, and a level that runs out of budget **demotes** its oldest entry to the next level instead of deleting
it. **Only the cold level deletes.** Each level is switched off by its own budget; with no warm level configured
the tree is byte-for-byte the current one.

The **installer** (new) adds two questions to the flow it already has - one request or several, and whether the
conversations are kept on disk - placed **after** the model and the machine bench and **before** the context size,
so the slot count is chosen with the model and the context already known. **The always-path does not change, and
that is measured rather than asserted:** `setup.py` before and after this change, across 5 machines x 5 sizes x
(`--yes` or Enter), gives **50 cases and 50 identical results**, with `--yes` asking nothing at all. No answer means
the previous behaviour.

### The problem, and why

With four parallel slots, long contexts and a bounded `--kv-resident`, host RAM runs out while a conversation is
being parked. When the physical-RAM admission gate failed, the engine **dropped the capture instead of spilling
it** — the disk tier sits *after* the gate — so the next turn **re-prefilled the whole prompt from token 0**.

Measured over a **49.5-minute window of real use** (2026-10-14, 19:32:30 → 20:22:03, the configuration before the
parking target):

| symptom | number |
| --- | ---: |
| rejected parks (`skip parking (physical RAM admission …)`) | **28** |
| disk copies ignored (`disk hit skipped`) | **23** |
| K/V re-processed | **1,955,069 tokens = 2,970,832 ms (49.5 min) of engine prefill time** |
| worst single episode | **141,718 tokens in 197.3 s** |
| host free RAM (1,277 samples @2 s) | min 904 · avg **4,070** · max 5,784 MiB |
| engine working set | avg 10,711 MiB (max 16,046) |

Two readings that matter:

- **The floor alone is almost never what blocks; `snapshot + floor` is.** Host free RAM sat below the 2560 MiB floor
  in only **65 of 1,277** samples (5%), while the real condition is `available ≥ snapshot + 2560` and a snapshot
  measures **1.6–2.9 GB** (working figure ≈2.5 GB per conversation). In **every** rejection the log reads
  `evicted 0, 0 still parked`: the RAM cache was **empty**, so there was nothing to evict — it was system RAM, not
  the cache's budget.
- **The rejections are policy, not lack of disk.** The disk tier was already **full** (19,855 / 20,480 MiB, 101 age
  evictions) and still spilling correctly; the RAM gate that precedes it is what refused.

Refusing did not even save RAM. A disk restore is indistinguishable from RAM — `disk/checkpoint` **95,661 tokens in
273.7 ms** against `live` **102,362 tokens in 267.6 ms** — and the re-prefill rebuilds in the slot the same K/V the
disk copy would have loaded (the final state is identical; `/metrics` reports the same `held_tokens`). The engine
refused to do in **274 ms** what it then did in **197 s**, and ended at the same consumption.

> These are the spec's extraction of the window. The eviction report's own extraction of the same window counted
> **24** rejections (the measurement was still running). Both figures are stated; neither is averaged.

## What changed

### The parking target and the two floors

- **`--conversation-cache-park ram|disk|auto`** (default **`ram`**) — the parking target.
  - **`ram`** — capture to host RAM, keep it, mirror it to disk (`spill-on=park`). What the engine did before.
  - **`disk`** — park to disk only: capture, hand the copy to the asynchronous spill writer, and **free the RAM
    copy at once**. The RAM cache keeps nothing for that park (`conversations.size()` does not grow; `put()` is not
    called for it).
  - **`auto`** — resolved **once at startup**, after the arena and the model are resident: `disk` when the free RAM
    cannot hold one snapshot per cache slot beside the floor, `ram` otherwise. Reported on stderr
    (`park target auto -> disk|ram`).
- **`--conversation-cache-park-min-free-mib`** (default **`512`**) — the **transient RAM floor** a disk-bound
  capture may take, as opposed to the **parking floor** `--conversation-cache-min-free-mib` (default **`2560`**) that
  a capture which *stays* in RAM must leave. Two floors because the big one exists to stop the OS paging during a
  capture that *stays* (that is the reason recorded in the code comment); a disk-bound capture is transient — its RAM
  is freed right after — so refusing it only pays the re-prefill. `0` = require only `need`.

**The transistor (the core rule).** In both paths (the park and the disk hit) and both modes: **when the big floor
does not hold, the capture is not discarded — it spills to disk under the transient floor.** "Refuse" and "spill"
stopped being mutually exclusive. `disk` needs the disk tier (`--conversation-cache-spill-dir` with
`--conversation-cache-spill-on park`); without it the parking **stays in RAM** with a warning rather than parking
nothing.

**The same rule governs every disk read and save**, in one function (`conversation_disk_placement`, the park rule
with the destination fixed to disk): the conversation **disk hit**, the **segment-tier hit**, the
**system-prompt-cache hit**, the streamed **session restore** and the **API session save** are admitted under the
transient floor when the parking floor does not hold, and refused only when neither holds — each printing
`… hit under the transient RAM floor (need … plus 512 MiB, not 2560 MiB)`. A copy already on disk is never ignored
for RAM it would not keep.

Measured motivation for `disk`: with 4 slots the RAM cache's committed parking is up to the **8 GiB** budget (two
parked conversations alone measured **4.51 GB**, at **1.6–2.9 GB per conversation**); in `disk` mode that committed
RAM becomes **reclaimable file cache** instead.

### The cascade: a warm level, and degrade instead of delete

The cache now runs **VRAM → RAM → warm → cold**. The **warm** level is new and opt-in; the **cold** level is the
disk tier this series already had.

- **`--conversation-cache-warm-dir`** + **`--conversation-cache-warm-mib`** (empty / `0` = the warm level is off).
  This is the new pair.
- **`--conversation-cache-cold-dir`** + **`--conversation-cache-cold-mib`** are the **canonical names of the
  existing disk tier** — the same field, the same directory, the same files. `--conversation-cache-spill-dir` and
  `--conversation-cache-disk-mib` keep working as **aliases of the same field**; two names pointing at one field
  with different values is a start-up error. The reason the cold level is the *existing* directory and not a second
  one: the tier the operator already runs is sitting on hundreds of GiB of parked conversations, and making the cold
  level the canonical name of that same field **inherits every one of them with zero migration** — the engine
  recognises a directory by the `strata-conv-` name prefix and the sidecar identity, not by which flag opened it,
  and it rewrites nothing on open.

**Degrade, not delete.** When a level's GC runs (`enforce_budget`, `enforce_age`) and a next level is set, the
entry is handed to the next level (**`degrade_to_next`** → **`adopt`**): the cold level writes its own sidecar,
relocates the session files (`relocate_file` — a rename on one volume, a copy+delete across two) and re-seals the
entry's age; the warm level then drops its own copy. **Only the last level (the cold) deletes** — FIFO by budget,
and by age if the operator asked for it. A degradation is **not** an eviction: the warm level's `disk_evictions`
and `age_evictions` counters do not move.

**All-or-nothing.** The entry is written to the cold level **first** and unindexed from the warm level **after**; if
the cold level cannot receive it (budget, `when-full=reject`, an oversized image, an I/O failure), nothing is moved
and the entry **stays in the warm level**. The whole move is reverted on any failure.

**The correctness paths still discard.** Only the two **capacity** levers (budget and age) degrade. Supersede,
compaction and an unreadable image still discard, exactly as they do today; a warm level with a cold level below it
does not change that.

**The RAM floors are not cascade levels.** `--conversation-cache-min-free-mib` and the transient
`--conversation-cache-park-min-free-mib` stay what they are; the cascade is a policy of the two disk levels and it
reuses the same transistor, adding no third admission rule.

**Off by default.** No warm level configured means no cascade at all: the write level is the cold level, the read
path says `from disk` and returns the same instance as before, and the tree is byte-for-byte the one that ships
without this change. The gate for a level is the same one the existing tier already uses (a non-empty directory
*and* a non-zero budget).

### The commit list

The series in order, oldest first: thirty commits on `296b59d2`, messages verbatim (no hashes by hand - `git log`
at the tip, published `2026-10-15`).

1. `07613f4b` - serve: durable disk tier for the conversation cache (spill, archive, system-prompt cache)
2. `c514173e` - serve: two-pass streaming session restore (read pass validates, apply pass copies 16 MiB at a time)
3. `26d29b52` - serve: per-segment disk tier for the conversation cache (content-addressed segments)
4. `7cd80013` - serve: anchor the segment frontier at the <|im_start|> opener, not the role token
5. `1cc359d4` - serve: prefix restore ladder when a parked image fails validation (STRATA_RESTORE_LADDER)
6. `3e14ba41` - cuda: reserve the expert arena before the PLE/GGUF, and Windows chunked large pages
7. `0269a3c2` - serve: continuous agenda for batched reads (STRATA_BATCH_AGENDA, opt-in)
8. `940eb9bf` - docs: document the disk tier, restore ladder, anchored frontier and chunked large pages
9. `acc09864` - serve: --conversation-cache-park (ram|disk|auto), and a rejected park spills to disk instead of discarding
10. `571f1203` - serve: the transient disk floor decides the sibling disk hits and the API session save/restore too
11. `2f8d3cad` - serve: default the conversation spill directory (--conversation-cache-spill-dir)
12. `20e5760b` - serve: expose the disk tier's folder and live state, and the admission source, to the Monitor
13. `b9b83c86` - serve: the Monitor shows the parking by destination, the per-slot speeds and the admission state
14. `d40cb783` - docs: document the parking destination and the resolved spill directory
15. `7b965cb3` - test: cover the admission source, the INFO line and the disk lifecycle in the Monitor
16. `16b67c46` - serve: expose the system-prompt cache's folder and budget in the INFO line
17. `1f572df4` - serve: expose the system-prompt cache's folder and budget, and stop reporting an absent counter as 0
18. `957d27b0` -