#2522 engine, server: #533 elastic VRAM per GPU of a layer split (VRAM <reserve_mib> [device])

本文

## What this changes

`#533` on a layer split: the VRAM reserve becomes per card.

- engine: `--vram-elastic` no longer refuses a layer split; every card's expert cache is segmented (512 MiB), and `VRAM <reserve_mib> [device]` moves one card's cache (device = the engine's number, 0 = the main card, the `--split-device` numbering) or every card of the run. Shrink/grow run on that card against its own free VRAM, the eviction scan is scoped to that stage's layers (slot numbers are per cache), and the prompt loan is re-lent at the largest chunk every card can still lend. One report line per card, then `VRAM-DONE`.
- verify: `refresh_residency` re-decides the 100%-resident fast path from the residency as it is now, and throws away the captured window graphs when it flips — the zero-doorbell capture holds no per-layer plan waits, so a shrink that takes a resident expert has to leave it, or the CPU pool is never told about the misses.
- server: `POST /v1/vram` takes `gpu` (validated against the run's cards), answers with one entry per card in `gpus` plus the totals over them, and still works with an engine that answers one line and no `VRAM-DONE`. `min_free_vram_mib` reads every card the run will use and names the busy ones.
- a lone dash names the start's reserve so a device can follow it (`VRAM - 1`): a number in the first slot IS the reserve, so `reserve_mib: null` with a `gpu` used to send `VRAM 1` — a 1 MiB reserve on EVERY card, not the start's reserve on GPU 1. `VRAM -1` is still refused.
- `shrink` keeps whole segments and rounds its keep UP, so a reserve asked for arrived up to one segment short and a need under one segment freed nothing at all; `segment_floor` rounds the keep DOWN to a segment boundary (12000 asked from 8292 free lands 12384, not 11872).
- SYCL port: the same command syntax and report lines; the segmented cache stays CUDA-only, so it refuses.
- `vram_elastic` and `vram_segment_mib` are editable keys (from the next start), and `tools/comfy_vram_handoff.py` drives the whole handoff: reserve, queue, wait, /free, restore.
- docs: the #533 paragraphs in DETAILS.md and MULTI_GPU.md.

## What it leaves alone

The default path is byte-identical to the last release: `vram_elastic` is opt-in, and with it off a split runs as before. Single-card runs, the helper caches, `--peer-device` and the resident low-RAM mode are untouched. The split itself never moves: a card keeps its dense weights, its session and its KV, and nothing is reloaded.

## Measured (2x RTX 3090, driver 595.71.05, Swift 1.5 IQ3_XXS, layers 0-23 / 24-47, `"parallel": 2`, `--vram-reserve-mib 700`, 512-token greedy decode)

- With ComfyUI's Qwen-Image 2.1 on the second card: GPU 1 went from 16380 to 1024 MiB of cache (559 of 8833 experts, the prompt chunk 8192 -> 512) and left 15200 MiB free, where the image model's measured peak at 1024x1024 with DynamicVRAM is 15240 MiB. With ComfyUI holding 15240 MiB on GPU 1 and that card's cache at 3072 MiB, the engine answered a request in 1.1 s.
- With the cache whole on both cards, `"vram_elastic": true` and `false` give the same cache (20 091 slots, 33 037 MiB) and 106.6 vs 106.4 tok/s — the segments cost nothing while nothing is handed away.
- A second CUDA client costs 305 MiB before it allocates anything (`torch.cuda.init()` and its first tensor took the card from 23505 to 23810 MiB used); it fails in `cuDevicePrimaryCtxRetain` with CUDA_ERROR_OUT_OF_MEMORY at 80 MiB free and attaches at 613.

## How I checked

- `serve/test_vram.py` on this branch: 14 tests, OK — the line with a device, per-card aggregation, the old engine, refusals, the route, a reserve kept for an unloaded engine.
- `serve/test_server.py` on this branch: 311 tests, OK — its free-VRAM guard takes the per-card reading and checks a two-card run.
- Built and measured on CUDA (the 2x 3090 above). The SYCL port carries the same syntax and refuses; it was not compiled here, and no HIP build was run.