#1841 serve: an opt-in VRAM headroom for --expert-cache auto (#1549)

本文

## What this changes

`--expert-cache auto` sizes the VRAM-resident expert tier to every byte the card reports free. On a card
that ends up nearly full that is wrong by a lot: the prompt path then reads tens of times slower while
doing exactly the same work.

It adds one opt-in setting and **changes no default**:

```
STRATA_EXPERT_CACHE_HEADROOM=<fraction>    # "1/6", "0.17", or "0" for the sizing you have today
```

the share of free device memory `auto` keeps for the session's other buffers instead of experts. The
helper is `strata::core::expert_cache_headroom()` in `include/strata/core/expert_cache.hpp`, read by
the CUDA/HIP sizing in `src/program/generate.cpp`.

**What it leaves alone.** The default is zero, so every backend keeps the sizing it always had and the
default path is byte-identical to the last release: with the variable unset, the old expression is
evaluated. An explicit `--expert-cache N` is the user's own budget and is never cut. When the headroom
would leave no room at all, the old sizing stands. Nothing in the prompt, decode or expert paths is
touched - the cache is just sized smaller.

## Why, and what it was measured on

An Arc Pro B70 (32 GB, driver 32.0.101.8976), the Qwen3.8-Flash-Next IQ3_XXS pack, OpenCL backend,
eager, `--prefill 4096`, greedy, 64-token prompt. The same 3,668 gate/up and 3,668 down naive GEMM calls
at every cache size - only the cache size and the memory left over change:

| expert cache | VRAM of experts | prefill | ms per expert GEMM call |
|---|---|---|---|
| 17,687 slots (`auto` before) | 28.82 GiB | 426 s | 111 |
| 15,382 slots | 24.95 GiB | 472 s | 122 |
| 14,704 slots | 23.87 GiB | 9.7 s | 2.32 |
| 10,698 slots | 17.36 GiB | 10.3 s | 2.39 |

A cliff, not a slope: every size up to 14,704 slots is stable at ~10 s and every size from 15,382 up
takes minutes. The two expert GEMM phases are 95.8% of the GPU timeline in the slow arm, and the phase
waiting for the host-to-device staging copies is **34 ms of 957,283** - the bytes are not the problem.
Memory during the slow pass peaks at 31.2 GiB of dedicated memory (the card full) and holds ~15 GiB
shared; the fast one peaks at 30.08 GiB dedicated and ~3.4 GiB shared. That is consistent with WDDM
paging an over-subscribed allocation to system memory, which the engine's own comment at the sizing
describes, but it is not proven.

Ruled out, each measured: the host mirror (35% of the same pack mirrored over PCIe costs nothing, 2.05
ms per call against 2.01 with every expert resident); the prompt length (the cost is fixed per expert
call, not per token); subnormals in the dequantized weights or activations (none in either pack);
`--kv-resident` (7,589 ms against 7,587 ms). The kernel calls and their shapes are identical in every
arm. No accuracy cost: the cut and the uncut sizing give identical greedy output tokens - eight tokens,
`271 71093 271 550 18381 198 12 1510`, from a scrambled 48-token prompt.

`docs/EXPERT_CACHE_HEADROOM.md` has the full findings, the A/B protocol for a card this has not been
measured on, and what each outcome would mean; `docs/TEST_REQUESTS.md` gets a row for it.

## How I checked it

- With the variable unset, `free_room` is the expression the last release evaluates - no code path
  changes.
- With `1/6` set on the B70, the knob reproduces the measurement: 14,688 slots (23.83 GiB) against
  17,687 (28.82 GiB) unset, with identical output tokens.
- The header compiles: the SYCL build includes it and passes.
- **Not compiled:** this machine has no `nvcc` or `hipcc`, so the CUDA/HIP sources in this pull request
  were not built where the change was written. The call site is the same statement the SYCL port makes
  against the same helper.

## What this is not

This is not a claim that the effect is real on NVIDIA or AMD. It is Windows/WDDM evidence from one
Intel card, offered as a knob so it can be measured elsewhere. If it turns out to be a no-op on CUDA
the default stays zero, and if the cliff is real there the numbers from the A/B are what would justify
turning it on.

Closes the sizing half of #1549.