#3265 [Benchmark] Experts in VRAM ahead of time: oracle run 3.0x prose, 4.1x code decode, same output

Descrição

## Problem and result
On this machine most of a decoded token's time is the CPU computing experts that are not in VRAM: 46.9 of 60.0 ms per token, with 69.7% VRAM cache hits (Swift 1.5 IQ2_XS, RTX 5070). This PR measures how much faster decode gets when the right experts are already in VRAM, as a ceiling for a possible lookahead expert prefetcher. Nothing in Strata changes.

Oracle run: the VRAM cache pre-filled with exactly the experts the continuation routes to, adaptive tier off.
- Prose, 32 tokens: 12.68 → **38.05 tok/s (3.00×)**; hits 58.4% → 88.5%
- Code, 24 tokens: 10.41 → **42.34 tok/s (4.07×)**; hits 46.8% → 88.4%
- All 12 runs' output IDs equal the greedy reference. A control arm (adaptive tier off, normal profile) equals the normal arm, so the gain is from which experts are in VRAM.

Also recorded: a perfect lookahead prefetcher would copy 39–84 experts per token at 16–64 tokens ahead (2–4 ms at the measured 28.4 GB/s host→device), distinct experts per window up to 64 tokens, and the prompt-path cost of 4- to 64-token chunks as an estimate of larger verify windows.

## Related discussion
Follows from the NCM study, #3247 (NCM predicts a rough picture of a block of future text; the experts a block uses are that kind of picture). No NCM model is used here. Benchmark report only; no feature or default change proposed in this PR.

## Approach
- `docs/EXPERT_LOOKAHEAD.md`: plain summary first, then settings, the five measurements, what it does and does not show, how to repeat.
- `docs/benchmarks/expert-lookahead-2026-10-19-rtx5070/`: every run's stdout/stderr, results JSON, the two routing traces and prompt/reference IDs used (inputs, 0.8 MB total), PCIe benchmark output, profile and engine SHA-256. Local paths rewritten.
- `tools/expert_lookahead_bench.py`: `unions`, `simulate` and `oracle-profile` run without a GPU on the committed traces; `oracle-runs` and `prefill-sweep` drive the engine.

## Validation
**Passed:**
- RTX 5070 (12 GB), Ryzen 9 7900X, 61 GiB, Linux 7.0.0-38, driver 610.43.02; existing engine 0.1.45 binary, checkout e88ee39d; saved run config with context 4096, KV residency 0, `--mmap-experts`, greedy reference mode (`--spec 2 --spec-corrupt 1`, no MTP).
- Oracle runs: 3 arms × 2 prompts × 2 repeats in alternating order; repeats within 3%; 12/12 output IDs equal the reference.
- Chunk sweep: 9 runs, repeats within 2%.
- The tool reproduces the recorded results: identical union and simulation output, byte-identical oracle profiles (SHA-256 in the PR), and the same engine argument lists as the recorded runs (all 12 oracle runs and the sweep, compared without running the engine).
- `git diff --exit-code main -- src include serve strata_cli strata_control apps requirements.txt` returns 0.

**Failed:**
none observed. A first cold T=8 chunk run (263.1 ms vs 221.8 ms) is kept separately and left out of the table.

**Not run:**
- A real prefetcher or predictor; copying experts during decoding (the oracle experts were loaded at start-up, 1.2 s, untimed).
- Long answers: only 24–32 tokens' experts fit in 4,866 slots. Normal arms are slower in short runs (79–96 ms/token) than over 64 tokens (60 ms), so the gain over long answers is likely smaller.
- The saved 64K configuration, MTP or prompt lookup on top, other machines, HIP/SYCL/Metal.
- No engine code changed, so no backend builds.

## Limits
Two prompts, one machine, diagnostic settings, short runs. This is a ceiling for a perfect predictor, not a speed-up of Strata. A real predictor's misses cost roughly 17 ms per token per 10% of routed experts missed (estimate from these runs).

🤖 Generated with [Claude Code](https://claude.com/claude-code)