#2963 resident RAM mode: the RAM copy is page-locked in slices when the one page-locked allocation is refused (32K prompt 9.9 -> 6.7 s, decode -7% a round)

Descrição

With `--resident-experts` (and `--resident-budget-gib`), the experts the GPU does not hold are copied into RAM, page-locked by one `cudaHostAlloc`. When the driver refuses that one allocation, the copy was malloc'd and only locked in the working set. A Q8_0 pack's ~100 GiB copy on a 128 GB Windows PC is such a case: WDDM caps page-locked memory at the GPU's shared-memory budget.

What the refusal costs today, in `pin_cache_complement`:
- `pinned()` is false for every expert and `pcie_layer()` is false, so the PCIe share (`--pcie-frac`) fetches nothing and every decode miss goes to the CPU.
- The prompt path copies every expert through the stager's host buffers instead of by DMA from the copy.

**Change.** Three paths, each named in the log (`page-locking path 1/2/3 ... X of Y GiB pinned`):
- **Path 1:** the one `cudaHostAlloc`, as before.
- **Path 2** (new): the allocation was refused, or has no device alias.
  - The copy is `VirtualAlloc`'d (`posix_memalign` elsewhere) and registered with `cudaHostRegister(Portable | Mapped)` slice by slice, before anything writes it.
  - Each slice is about one layer of the pack, cut at expert starts (page-aligned where possible), with its own device alias. Registration stops at the first refusal.
  - `pinned()` and `device_alias()` look the slice up. An expert that no slice holds whole is the CPU's. `pcie_layer()` is true.
  - By default the slices stop below the GPU adapter's shared-memory budget, less what the process uses there and a margin. On a layer split, the smallest room of the devices counts. Pinned until the driver refused a slice, the copy filled that WDDM segment, and every GPU phase of the prompt path got slower (numbers in the code comment).
  - `STRATA_COMPLEMENT_PIN_GIB`: `all` = slices until the driver refuses one, `N` = at most N GiB, `0` = path 3.
- **Path 3:** no slice pinned. Pageable and locked, as before. The `STRATA_PARTIAL_PIN=1` budget prefix keeps its own path.
- `pinned.cu`: #243's private `sliced_pin_limit` becomes the public `shared_pin_limit_for(dev)`. The arena's `STRATA_ARENA_PIN_GIB=auto` calls it for the current device, as before. `STRATA_PIN_MARGIN_GIB` sets the margin (default 4 GiB, #243's value).
- `STRATA_TEST_COMPLEMENT_REFUSE=1` takes the one allocation as refused. This lets paths 2 and 3 be tested on a PC whose driver grants it.
- `docs/DETAILS.md`: one bullet in the resident-mode section.

**Identity** against 477ac13c: first-token logits (sha256) and greedy tokens, `--pcie-frac 0 --adapt-every 1000000 --expert-cache 3000`.

| run | p2k | 32K |
|---|---|---|
| 477ac13c, default mode | `1e9c612c` | `f5b2872c` |
| this PR, default mode | identical | identical |
| 477ac13c, `--resident-experts` | same as 477ac13c default | same |
| this PR, `--resident-experts`, path 1 | identical | identical |
| this PR, path 2 (`STRATA_TEST_COMPLEMENT_REFUSE=1`; 45 of 45 slices, 58.97 GiB) | identical | identical |
| this PR, path 3 (also `STRATA_COMPLEMENT_PIN_GIB=0`) | identical | identical |

**Speed.** Setup:
- RTX 5090, Ryzen 9 9950X3D, 128 GB, Windows 11, CUDA 13.3, portable release build.
- The GPTQ NVFP4 pack (the `nvfp4` family's files), `--resident-experts --expert-cache auto --vram-reserve-mib 1500 --prefill auto --spec 6 --spec-min-p 0.7 --kv int8 --max-context 262144`. The cache gets 7,157-7,279 slots and the RAM copy is 48.15 GiB.
- This PC grants that copy, so paths 2 and 3 are forced with `STRATA_TEST_COMPLEMENT_REFUSE=1`.
- 3 runs per path, interleaved. Each run: 3 decodes of up to 600 tokens after a 45-token prompt, then two fresh 32K prompts.

| | path 3 (today's fallback) | path 2 (this PR) | path 1 (allocation granted) |
|---|---|---|---|
| 32K prompt | 9.93 ± 0.32 s | **6.74 ± 0.09 s** | 6.67 ± 0.01 s |
| decode, ms a round | 20.2 ± 0.7 | **18.8 ± 0.4** | 18.3 ± 0.4 |
| decode, tok/s | 120.0 ± 3.1 | **130.2 ± 2.6** | 133.6 ± 3.3 |
| ready (process start to engine ready) | 21.2 ± 0.5 s | **15.6 ± 0.2 s** | 15.4 ± 0.2 s |

Path 2 registered 37 of 37 slices (18,699 of 18,699 experts) in 3.4 s. Path 3's ready time includes locking 48 GiB in the working set.

Earlier, on our fork's code with a Q8_0 pack, where the driver really refuses the allocation (a ~100 GiB copy, same PC): decode 48.8 -> 53.3-55.0 tok/s, 32K prompt 9.9 -> 8.1-8.5 s.

A first batch without `--vram-reserve-mib` (the cache filled the card, ~100 MiB free) had one path-2 run at 22 s for the 32K prompt, with the same log and the same expert counts as the others. The engine's own "only N MiB of VRAM free" warning names that state: WDDM pages GPU memory. A small probe on this PC copies 2.6 MiB blobs at 56.3 GB/s from 4 KiB pages registered in slices, the same speed as from `cudaHostAlloc`.

**How to reproduce:** install with upstream setup `--family nvfp4 --model NVFP4`. Add `--resident-experts --vram-reserve-mib 1500` to the engine arguments and start it with `STRATA_TEST_COMPLEMENT_REFUSE=1` for path 2, or with `STRATA_TEST_COMPLEMENT_REFUSE=1 STRATA_COMPLEMENT_PIN_GIB=0` for path 3. The startup log names the path. Then read a 32K prompt fresh (we ran `--spec 6 --spec-min-p 0.7`; setup writes 4 / 0.5).

**Not measured:** Linux and HIP (path 2 registers the same way there; not run), a layer split with the resident mode, and a real refusal on 0.1.45 (no Q8_0 pack on this PC now).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01VZy1yKaDDiA8a7svdwaHio