#1903 Arena mode, one GPU on Windows: ~3 ms per window in the blocking adaptive tier, and layers 32-47 never get a PCIe share (RTX 3090 + Ryzen 5 5600X)

Descrição

**Machine.** RTX 3090 24 GB (sm_86, 350 W, also drives the desktop), PCIe 4.0 x16, Ryzen 5 5600X (6 cores, AVX2), 2 x 32 GB
DDR4-3600, Windows 11. Qwen3.8-Flash-Next GSQ-RCO IQ3_S, `--max-context 262144 --kv int8 --kv-grow --vram-reserve-mib 1200`,
`--spec 4 --mtp`, calibrated adaptive tier (`--adapt-every 1 --adapt-swaps 80 --adapt-decay 0.97`). The default arena: every
expert in RAM, 8,289 expert slots in VRAM. Strata v0.1.42 (`61b3fb5d`); local builds with VS 2022 17.14 + CUDA 12.8, sm_86.

### 1. The blocking adaptive tier: ~3.2 ms of a 25.9 ms window (same hole as #1863)

`STRATA_DECODE_TIMING=1`, a cold short prompt: 25.9 ms per window = verify 21.0 + commit/emit 0.1 + draft 1.6, so ~3.2 ms
are outside the breakdown, as #1863 measured on its machine. In the arena mode the tier's thread is created and joined
every window (`generate.cpp` ~11630-11685) and the next window waits for its copies (`apply_pending(true)` →
`cudaEventSynchronize(adapt_ev)`, then a synchronous `res_upload`). About a third of the swaps come from the part of the
arena that is not page-locked (below), so their `cudaMemcpyAsync` is synchronous. `STRATA_ADAPT_LAG=2` does not help here
(it halves the swap rate: −8 % cold). `--adapt-async 1` would, but it is refused outside the resident RAM mode.

### 2. Windows page-locks the first ~30 GiB of the arena only: layers 32-47 get no PCIe share

`cudaHostRegister` of the whole 47 GiB arena fails (WDDM shared budget, about half the RAM), the sliced fallback registers
the first 32 slices (30 GiB, layers 0-31), and `pcie_layer()` is false for layers 32-47: their misses are 100 % CPU, while
~10 GiB of the registered prefix hold experts that are GPU-resident and never read by DMA. Also `m = (nmiss * pcie_num) >> 8`
rounds the PCIe share to 0 at 1-2 misses per layer, which is the common case at a 88-95 % hit rate.

### Measured (cold A/B, paired)

12 distinct short prompts (FR/EN, code, prose), each sent once per fresh engine with a fixed seed, temperature 0.7, 256
tokens, 2 rounds with the arms' order reversed; per-prompt ratio against the reference arm.

| Arm (local builds of the PR branches against the v0.1.42 tag) | Decode tok/s (median) | vs v0.1.42 | Prompts faster |
|---|---:|---:|---:|
| v0.1.42 | 97.2 | 1.00 | — |
| `--adapt-async 1` allowed in the arena (copy-in only) | 107.7 | **x1.07** (x1.11 in an earlier run) | 12/12 |
| pin by need + `STRATA_PCIE_MIN1=1` + `--pcie-frac 0.6`, blocking tier | 99.3 | x1.00 | 6/12 |
| the same on top of `--adapt-async 1` and other opt-in switches | — | x1.02-1.03 | 10/12 |

With `--adapt-async 1` the window drops to 21.6 ms (adapt wait 0.41 ms, join 0). Pin by need: 28.96 GiB registered = 92 % of
the non-resident experts, every layer gets a PCIe share; the prompt read is 5-7 % faster at 56K-117K tokens (2,172 → 2,313
and 2,091 → 2,208 tok/s).

A pitfall for anyone measuring this: a prompt sent twice in a row runs ~25 % faster the second time (the tier has adapted to
it); an A/B that always sends arm B second shows a gain that is not there.

### What I will send

Two pull requests, each opt-in with the default path byte-identical to v0.1.42:
1. the asynchronous adaptive tier in the arena mode (one GPU), with the K/V-grow and `VRAM` fences it needs;
2. pin by need (`STRATA_PIN_BY_NEED=1`, page-aligned arena stride) and the PCIe share floor (`STRATA_PCIE_MIN1=1`).

Raw data, harness and the full write-up: https://github.com/RubenTorney/Strata/tree/perf/async-adapt-arena
(`bench/results/2026-10-11-rtx3090-5600x-async-arena`); each PR carries its own report.

**Not tested:** HIP and SYCL builds, Linux, layer splits / several GPUs, the resident RAM mode with these builds (the new
paths are gated on the arena and one GPU; the K/V-grow fence also applies to the resident mode's asynchronous tier).