#2964 decode: STRATA_ADAPT_FETCH=1|2 (opt-in) - the PCIe share's most-routed misses are admitted into the VRAM tier (-5.3% a round on the NVFP4 pack)

说明

Decode reads expert blobs from host RAM three ways: the CPU pool, the PCIe share (`--pcie-frac` of a layer's misses, staged into VRAM by `fetch_blobs`), and the adaptive tier's swaps. The PCIe share's blobs cross into VRAM anyway. This opt-in lets them stay there.

**Change** (opt-in, `STRATA_ADAPT_FETCH=1|2`; unset changes nothing):
- The share takes the layer's most-routed misses, by the tier's decayed counts. Today it takes the last ones in routing order. Tail-skipped and unpinned blobs are not picked.
- A fetched expert can be **admitted**. It must be routed at least `STRATA_ADAPT_FETCH_MIN` (1) and `STRATA_ADAPT_FETCH_MARGIN` (0.5) more than the layer's least-routed resident expert that the window does not route. Then `fetch_blobs` copies it straight into that expert's slot (a new optional `dst2` target per blob) instead of staging, and it becomes resident: a swap with no RAM read of its own.
- `=2` leaves the tier's own swaps out of a layer where an admission ran since the tier's last round; the tier keeps the layers the share did not reach. `=1` runs beside them.
- The admissions write `host_res` during a window. The device table follows through #1517's pre-launch patch (3f2db5e2), whether the tier waits or not.
- The router lookahead copies the residency rows it reads when the layer is submitted, because the table now changes while the lookahead runs. It keeps the stats and foresight parts as they are.
- Experts the tier is copying in are left alone.
- It is off, with a log line saying why, in these cases: `--batch`, `--pipeline-windows`, `--adapt-async`, a layer split, peer or helper GPUs, the resident RAM mode, `STRATA_ADAPT_OVERLAP=1`, the foresight swap space, `--pcie-mode dma|direct`, or no PCIe share.
- `STRATA_ADAPT_FETCH_VERIFY=1` (`strata generate`) reads every admitted slot back and compares it with the experts' bytes.
- The verify plan block grows by 16 u64 (`dst2`). They stay zero unless the opt-in writes them.
- `docs/DETAILS.md`: one bullet next to #1517's.

**Unset = 477ac13c.** First-token logits (sha256) and greedy tokens are identical, with `--pcie-frac 0 --adapt-every 1000000 --expert-cache 3000`: p2k `1e9c612c`, 32K `f5b2872c`.

**Checks with it on** (`strata generate`, 600 tokens, `STRATA_ADAPT_FETCH=2 STRATA_ADAPT_FETCH_VERIFY=1 STRATA_ADAPT_CHECKRES=1`):
- 6,221 admissions. The first 6,159 were read back, and 0 differed.
- The residency check covered 266 windows: 0 with mismatches, 0 layer-windows with CPU experts and no activation published.

**A/B.** Setup:
- RTX 5090, Ryzen 9 9950X3D, 128 GB, Windows 11, CUDA 13.3, portable release build.
- The GPTQ NVFP4 pack through serve: `--expert-cache auto --vram-reserve-mib 1500 --pcie-frac 0.25 --prefill auto --spec 6 --spec-min-p 0.7 --kv int8 --max-context 262144`.
- The cache gets 7,185-7,221 slots, about 29% of the 24,576 experts; the adaptive tier is on, as by default.
- The same build in both arms, 4 interleaved pairs. Each run: 4 decodes of up to 600 tokens after a 45-token prompt.
- "ms a round" = decode time / (tokens - accepted drafts). It does not depend on how many drafts a run accepts.

| | unset | `STRATA_ADAPT_FETCH=2` | |
|---|---|---|---|
| ms a round (per run) | 16.31 / 15.73 / 15.99 / 15.97 | 15.26 / 15.04 / 15.10 / 15.22 | |
| ms a round (mean) | 16.00 ± 0.24 | **15.16 ± 0.10** | **-5.3%**, every pair faster |
| decode tok/s | 151.9 ± 4.2 | 159.2 ± 1.0 | +4.8% |

Earlier, on 0.1.40.3 with ISTA-DASLab's IQ2_XS model (a 6,297-slot cache, `--pcie-frac 0.25`): chat -9.1% and 32K explain -6.8% a round. That model is no longer on this PC.

**How to reproduce:** install with upstream setup `--family nvfp4 --model NVFP4`. Add `--vram-reserve-mib 1500 --pcie-frac 0.25` to the engine arguments and compare decode with and without `STRATA_ADAPT_FETCH=2`. The startup log has the `STRATA_ADAPT_FETCH=2: ...` line. We ran `--spec 6 --spec-min-p 0.7`; setup writes 4 / 0.5.

**Not measured:** `STRATA_ADAPT_FETCH=1` on this pack, other cache sizes, HIP, and Linux.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01VZy1yKaDDiA8a7svdwaHio