#2968 prompt path: experts staged from the resident arena keep the full streamed ring and 8 copy threads (unpinned 32K +5-16%); STRATA_ARENA_PIN_GIB=none

Description

Some experts the GPU cannot DMA directly: the pin is capped or refused (a multi-GPU run under WDDM, `STRATA_ARENA_PIN_GIB`, the remainder `STRATA_PIN_BY_NEED` leaves). The prompt path's stager copies these into pinned buffers and DMAs them from there.
- When more than 10% of the bytes are unpinned, the streamed ring is cut to a quarter (96 slots) and the stager keeps 4 copy threads and 16 buffers.
- The 96 slots were measured where the host copies were the limit: file reads, IQ3_S on a 64 GB PC.
- From the expert arena, these copies are plain memcpy from resident RAM.

**The change.**
- When the source is the arena (`Prefill::set_resident_stage`, set where the engine sets the pinned share):
  - the ring keeps its pinned size (384, or 1024 fused);
  - the stager gets max(4, min(8, CPUs/4)) threads and 64 buffers.
- Unchanged: the low-RAM file tier, and a fully pinned arena.
- Overrides:
  - `STRATA_STAGE_RESIDENT=0` restores the old sizing; `=1` forces the new one.
  - `STRATA_STAGER_THREADS`, `STRATA_STAGER_RING` and `STRATA_PREFILL_RING` still take precedence.
- It logs one line: `prefill: 100% of the expert bytes are not pinned: the stager copies them from resident RAM with 8 threads into 64 pinned buffers (169 MiB) and the streamed ring keeps its pinned size ...`.
- `STRATA_ARENA_PIN_GIB=none` pins nothing: the unpinned arm, by name.
  - `0` keeps meaning "the whole arena", as documented. Until now, nothing pinned needed a 1 GiB cap below a layer's 1.32 GiB slice.
  - `none` is a one-byte cap, so no slice is registered. The arena stays resident through the working-set lock or its large pages.
  - Logged: `STRATA_ARENA_PIN_GIB=none: nothing of the expert arena is pinned`.

**VRAM.**
- No new device allocation. The larger ring is borrowed from the expert cache during a prompt and refilled after it, as today.
- Unpinned, the prompt path now borrows 288 more slots: 1615 instead of 1327 at 8K chunks, 5118 instead of 4829 at 32K chunks. That is **+0.74 GiB** (288 x 2.76 MB NVFP4 blobs) during a prompt.
- With `--no-prefill-borrow`, that +0.74 GiB stays reserved for the session.
- The expert cache's auto size came out the same within run-to-run noise (6923-7071 slots in both builds).
- Pinned host memory: 64 stage buffers = 169 MiB, up from 16.

**Measured on this PC.**
- Builds: 477ac13c (upstream/main, v0.1.45) vs this branch, the same configure (MSVC 2022, CUDA 13.3, portable).
- Machine: RTX 5090 (PCIe 5, 57.9 GB/s probe), Ryzen 9 9950X3D, 128 GB DDR5 (dual channel), Windows 11.
- Model: the NVFP4 GPTQ pack. One GPU, `--expert-cache auto --vram-reserve-mib 2048` (at the default 700, both builds had 0-275 MiB of VRAM free and WDDM paged).
- Two fresh 32K reads and three fresh 2K reads per start (prompt tok/s). Two rounds per arm, the order alternating.
- "Nothing pinned": on 477ac13c it is `STRATA_ARENA_PIN_GIB=1` (no slice fits), on this branch `=none`. Both log 0 slices pinned.

| arm | 477ac13c 32K | this PR 32K | 477ac13c 2K | this PR 2K |
| --- | --- | --- | --- | --- |
| nothing pinned | 3124 / 3203, 3226 / 3293 | 3245 / 3473, 3412 / 3469 | 951-965 | 925-957 |
| 8 GiB pinned (88% not pinned) | 3364 / 3432, 3276 / 3282 | 3653 / 3723, 3658 / 3733 | 1003-1028 | 999-1024 |
| all pinned (unchanged path) | 5882 / 5950 | 5906 / 6120 | 1875-1919 | 1887-1923 |
| nothing pinned, `--prefill auto:32768` | 5210 / 5677, 5212 / 5587 | 6034 / 6576, 5578 / 6020 | 961-968 | 924-961 |
| all pinned, `--prefill auto:32768` | 6834 / 7405 | - | 1902-1938 | - |

- The 32K second reads gain:
  - default 8K chunks: +5-8% nothing pinned, +9-14% with 8 GiB;
  - 32K chunks: +8-16% nothing pinned. That is 75-77% of the pinned 32K before and 81-89% after.
- At default chunks each 8K chunk streams every expert again. The staged path is then bound by DDR5: a staged byte crosses DRAM three times (read, write, the DMA's read). So the ring matters less there.
- 2K stays RAM-bound: about -2% nothing pinned (mean 955 -> 938), 0% at 8 GiB. The fork measured the same at 2K (1601-1607 -> 1566-1592).

**Earlier numbers.** These are from the strata-nvfp4 fork, 0.1.43.1 code, with 32K prompt chunks by default there.
- RTX 5090, same PC, 32K reads:
  - nothing pinned: 6065-6434 -> 7402-7965 (70-71% -> 87-88% of pinned);
  - 8 GiB: 6365-6673 -> 7717-8225.
- Rented RTX 4090 (Linux, PCIe 4, DDR4), one GPU, nothing pinned: 32K 3153 -> 4363, 2K 554 -> 824. That run also had the fork's Linux fix that moves the stager's copy threads off the host CPU, which is not part of this PR.

**Identity.**
- Default settings with the whole arena pinned, on the NVFP4 GPTQ pack: the first-token logits and the greedy tokens (32 max; both stop at the end of turn) for a 2K and a 32K prompt are byte-identical to 477ac13c's.
- The run used `--expert-cache 3000 --pcie-frac 0 --adapt-every 1000000` and `STRATA_PREFILL_CPU_SHARE=0 STRATA_PCIE_BALANCE=0 STRATA_IQ_MT_MIN=1`.
- Hashes: 2K `1e9c612c2e6c7a32...`, 32K `f5b2872c7ac30b17...`.

**Not measured:**
- a real multi-GPU PC under WDDM (one card here);
- `STRATA_PIN_BY_NEED` with a partial pin.

**How to reproduce.**
- Install with upstream's `python setup.py --family nvfp4 --model NVFP4` (Maximilian228/Qwen3.8-Flash-Next-NVFP4-GPTQ-Strata).
- Start `strata --serve` with the model arguments setup writes plus `--expert-cache auto --vram-reserve-mib 2048 --prefill auto --spec 6 --spec-min-p 0.7 --max-context 262144 --kv int8` (add `--prefill auto:32768` for the 32K-chunk rows).
- Env: `STRATA_ARENA_PIN_GIB=none` (this branch) or `=1` (main), or `=8`.
- Send a 32K prompt twice and a 2K prompt three times, each fresh (a distinct first token). Read `prompt ... read in N ms (X tok/s)` from the engine log.
- All of it was measured on that pack with upstream/main builds.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01VZy1yKaDDiA8a7svdwaHio