#2399 `--vram-elastic` with `--kv-grow`: VRAM takes its chunks where the K/V grows into (resubmits #2201)

本文

## Title
`--vram-elastic` with `--kv-grow`: VRAM takes its chunks where the K/V grows into (resubmits #2201)

Issue: follows up #2201 (held for #2189; "send it together with the new #2189"). Builds on #2398 for HIP.

## Summary

Today the two switches exclude each other. With `--vram-elastic` the engine says `--kv-grow is off` and puts the whole
K/V in VRAM, because #533's cache is cut into segments while `--kv-grow` needs one VMM range of chunks. On an R9700
with a 262K context that costs ~1,400 cache slots, more VRAM than a game or an image encoder would borrow. This PR lets
both switches run together.

This branch is #2398 plus one commit; only the last commit is this change. It does not depend on #2398 on CUDA, and
once #2398 is in, it rebases onto main with no conflict in the code.

## What changed

`src/program/generate.cpp`:
- With both switches, the cache keeps `--kv-grow`'s arena (one range of 2 MiB chunks). `VRAM` works on the region the
  K/V grows into: slots `[lo, top)`, just below the prompt path's loan.
- **Shrink:** the same eviction the K/V's growth uses (hotter experts there move into the coldest slots of the loan
  region first). It is now `kvg_take`, split out of `kvg_ensure` unchanged; the chunks go to the driver instead of to
  the K/V.
- **Grow:** new chunks are mapped there, and those slots are refilled with the profile's hottest experts that the GPU
  does not hold. This is now `kvg_give_back`, split out of `kvg_trim` unchanged. So after a grow the cache holds what a
  K/V trim would leave, not necessarily the experts the shrink took (the segments put the same ones back). The docs say
  so.
- The prompt path's loan and chunk are not touched. While the cache is shrunk, the K/V keeps growing from the cache's
  slots, not from the reserve.
- A small counter keeps track of the chunks given to the driver. It is settled against the arena's empty places,
  because the K/V's first cells are new VRAM, and a trim below them maps those chunks into the arena too. Without that,
  the count drifted by ~100 MiB and a grow reported "no VRAM" for chunks that had no place.

`docs/DETAILS.md`, `docs/VRAM_ELASTIC.md`: the new "With `--kv-grow`" section. The image-encoder section (#2196) no
longer says that `--vram-elastic` turns `--kv-grow` off.

What it leaves alone: either switch alone (`kvg_ensure` and `kvg_trim` do the same steps through the two helpers;
without `--kv-grow`, `VRAM` keeps the segments). The default path does not reach any of it.

## Extra Notes

**How it was checked**

On v0.1.44 (this branch: #2398 + this commit; R9700 32 GB, ROCm 7.14.1, Unsloth UD-IQ4_XS, every expert in RAM, 262K
context, `--kv-grow --vram-elastic` through `serve.server`):
- Full HIP ctest: 111 of 112. The failure is `ple_parity`, which needs the Q2_0 fixture.
- Live:
  - The cache was 10,940 slots (24.68 GiB), with `elastic K/V: 16384 of 262144 cells in VRAM`.
  - `VRAM 4000` left 4,000 MiB free (9,427 slots holding experts).
  - A 30.6K-token prompt with a needle was answered right while the cache was shrunk; the K/V grew from the cache's
    slots.
  - `VRAM 0` brought back 10,848 slots; the K/V keeps the rest for its cells.
- With setup's ROCm 7.10 wheels, #2398's check turns both switches off, the line says why, and the engine served
  normally.
- CUDA: builds and links with nvcc 13.4.92, sm_120, 0 errors. Not run on an NVIDIA GPU.
- SYCL: `sycl/` builds with oneAPI DPC++ 2026.1, 0 errors (the port has its own `generate.cpp`; this change does not
  reach it).

Measured before the rebase (0.1.43.1 + #2189, the same box), with a needle at 3/5 of each prompt and a 320-token answer:

| step | VRAM call | slots holding experts (of 10,865) | free | needle / decode |
|---|---|---|---|---|
| 78K prompt, whole cache | - | 10,492 (the K/V took 373) | - | ok / 96.6 tok/s |
| `reserve 4000` | 106 ms | 8,872 | 4,000 MiB | - |
| 78K prompt (the K/V grows again while shrunk) | - | - | - | ok / 92.0 |
| `reserve 8000` | 113 ms | 7,094 | 8,000 MiB | - |
| 156K prompt (the K/V to 2.13 GiB) | - | - | - | ok / 87.8 |
| `reserve 0` | 418 ms | 10,865 | 453 MiB | - |
| 78K prompt, then a short one (the K/V trims) | - | - | - | ok / 96.1 |
| `reserve 4000`, `reserve 0` | 101 ms, 199 ms | 9,281, 10,865 | 4,001, 453 MiB | - |

- The driver's own counter followed: 31.5 GiB, then 30.5 at `reserve 4000`, 28.0 after the 78K prompt, 24.1 at
  `reserve 8000` with the 156K K/V, and back to 31.4.
- `--kv-grow` alone: every needle was found, and the K/V grew and trimmed as before.
- With the elastic image encoder (#2196, now in 0.1.44) on the same card:
  - the first picture made `VRAM` keep 2 GiB free (52 ms), and the encoder started in 3.1 s;
  - a 156K prompt grew the K/V to 163,840 cells while the encoder held its share (needle found);
  - after 60 s without a picture the encoder stopped, and the cache was whole again.

**Not tested:** NVIDIA at run time, Windows, more than one shrink-grow cycle while the K/V holds a parked conversation.

🤖 Generated with [Claude Code](https://claude.com/claude-code)