#2583 sycl: adapt streamed expert cache with synchronous mirror exchanges on Battlemage

Descrição

## Summary

NoHost streamed-expert serving disables adaptation because an eviction would leave an expert in neither VRAM nor the mirror. Count expert presence in completed verify windows and synchronously exchange incoming mirrored experts with outgoing VRAM slots before updating both tables.

Dispatch reuses upstream `intel_gpu_gen` and selects the optimized implementation on `IntelGpuGen::Battlemage`, subject to the existing shape, alignment and configuration checks. Hardware measurements below are from B70.

Single-device, single-slot serving with NoHost device planning, streamed native experts, a complete mirror, fixed capacity, positive adaptation interval and 1–96 swaps, no prefill borrowing, no elastic cache and no profile saving. Use the existing adaptation settings; CLI, batch serving and unsupported configurations retain upstream behavior. Allocate usage counters only for the selected serving controller, after verifier initialization and before graph capture.

## What changed

`sycl/include/strata/core/gguf_expert_source.hpp`, `sycl/include/strata/core/verify.hpp`, `sycl/src/core/gguf_expert_source.cpp`, `sycl/src/core/verify.cpp`, `sycl/src/program/generate.cpp`.

## Adaptation controls

| Control | Unset (default) or `1` | `0` |
|---|---|---|
| **Master:** `STRATA_SYCL_MIRROR_ADAPT` | Enable mirror-aware adaptation with synchronous expert exchanges in the supported serving configuration. | Restore upstream behavior; adaptive swaps are disabled in the guarded NoHost streamed-expert configuration. |
| **Usage policy under mirror adaptation:** `STRATA_SYCL_ADAPT_WINDOW_PRESENCE` | Count each routed expert once per completed verification window. | Count every routing occurrence instead; mirror adaptation remains enabled when its master and configuration guards permit it. |

For full fallback, set `STRATA_SYCL_MIRROR_ADAPT=0`; the presence control then has no effect. The existing `STRATA_VERIFY_NO_HOST=1` and `STRATA_VERIFY_DEVICE_PLAN=1` settings and the configuration guards above remain prerequisites.

Set these variables before starting the engine and restart it when changing them: the master is read during initialization, and the presence policy is cached on first use. They support diagnosis and fallback; normal operation needs no setting for either new control.

## Current revision checks

Checked revision: `27a48ccee1472a593c97b38afe0ee2c6cc787d6f`.

The current revision builds in the same omix environment. Relevant native and guarded correctness checks pass. Fresh serving runs with 4096, 8192 and 32768 input tokens each produce 128 output token IDs identical to pure upstream, both with default dispatch and with the optimization disabled. Raw route-count accumulation also passes these serving cases. These profiled runs check correctness and activation; they are excluded from latency measurements.

Offline device-image compilation of the affected kernel units passes for BMG G21, DG2 and PVC. These compilation checks do not establish runtime correctness or performance on those devices.

The validation and performance results below were collected at revision `3847868239442eb6cb3c6186ac7199bf18d90105`. These timings apply to that measured revision; the current revision has separate correctness checks.

## Validation at the measured revision

One Arc Pro B70 32 GB, Linux/xe, Xeon w5-3435X; `intel/omix:0.4.0-devel-ubuntu24.04`, oneAPI DPC++ 2026.1.1, compute runtime 26.31.39395.13, Level Zero loader 1.32.0. Container limit: 96 GiB RAM / 16 CPU quota. All builds and GPU tests run inside this container.

Build: `cmake -S sycl -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_C_COMPILER=icx -DCMAKE_CXX_COMPILER=icpx -DSTRATA_SYCL_AOT=bmg-g31 -DSTRATA_SYCL_PARITY=ON`, then `cmake --build build --target strata` and the relevant parity targets.

resident_plan_parity, verify_parity and real expert exchange checks pass. Invalid exchanges leave both sides unchanged; guard regions, mirror complement and capacity remain intact.

Ten locally defined requests match pure upstream token IDs at each max-context setting, 32768 and 131072 (20/20). Cases cover arithmetic, Python list merging, prose, Chinese, JSON, Unicode, bug fixing and exactly 4096/8192-token synthetic Python review inputs. Context capacity and input length are separate axes.

A frozen 4096-token CLI input with `--pcie-frac 0` produces all 248320 first-token logits byte-identically to pure upstream; all values are finite. This correctness run is excluded from latency measurements.

The CLI check exercises the original fallback, not serving mirror adaptation. The serving identity suite and exchange checks cover the selected mirror path.

Profiling confirms that the optimized paths are exercised. Profiled runs are excluded from latency measurements.

## End-to-end performance

Timing comparison: candidate `384786823944` versus upstream `296b59d2f73c`, on the B70 environment described above. The current head (`27a48ccee147`) has been checked for correctness but has not been remeasured for endpoint latency.

TTFT is the time from the request to the first nonempty SSE token. TPOT is the time between the first and last nonempty SSE tokens divided by generated tokens minus one. Negative changes mean lower latency.

| Input / output tokens | TTFT baseline → default (s) | Change | TPOT baseline → default (ms/token) | Change |
|---|---:|---:|---:|---:|
| 4096 / 128 | 3.473033 → 3.464599 | -0.243% | 14.320845 → 13.447906 | -6.096% |
| 8192 / 256 | 7.027440 → 7.008944 | -0.263% | 13.604901 → 12.966669 | -4.753% |
| 32768 / 256 | 29.697403 → 29.643881 | -0.180% | 14.078699 → 13.462772 | -4.351% |

With these same requested settings, upstream disables adaptive swaps under NoHost; the candidate keeps adaptation active using the synchronous mirror exchange. The endpoint difference includes the resulting cache-policy change, not just exchange-kernel execution time.

<details>
<summary>Test configuration and sampling</summary>

| Item | Setting |
|---|---|
| Timing baseline / frontend | `296b59d2f73c8f04a805df29c2fa97655adace78` |
| Timed candidate | `3847868239442eb6cb3c6186ac7199bf18d90105`; this PR only |
| Model | Qwen3.8-Flash-Next-GSQ-RCO, IQ2_XS; revision `ed59f92082b1e93c0e96d60a8b11aab089b52f09` |
| Sampling | Greedy, seed 42, thinking off |
| KV and context | INT8 KV; max context 131072; resident KV 32768 |
| Expert cache | `--expert-cache 17772` experts; reserve 1536 MiB; mirror budget 16384 MiB |
| Prefill and prompt cache | Auto prefill; no prefill borrowing; prompt cache off |
| Speculation and adaptation | spec4 / min-p0.5 / suffix3; adapt-every8 / decay1 |

Observed capacity in every arm: 18609 resident expert slots and 5967 mirrored experts (8.02 GiB). The 16384 MiB setting is a budget, not the actual mirror allocation.

Existing upstream settings held equal in both arms: `STRATA_VERIFY_DEVICE_PLAN=1`, `STRATA_VERIFY_NO_HOST=1`, `STRATA_STAGER_THREADS=12`, `STRATA_MIRROR_MIB=16384`, `STRATA_DECODE_TIMING=1`; `ONEAPI_DEVICE_SELECTOR=level_zero:gpu`, `SYCL_CACHE_PERSISTENT=0`.

Each arm starts a fresh process. One short request and three full-input requests are excluded as warmups; three requests are then measured.

| Input / output tokens | Arm order | Measured requests: baseline / candidate |
|---|---|---:|
| 4096 / 128 | Five baseline/candidate pairs, alternating order | 15 / 15 |
| 8192 / 256 | Baseline / candidate / baseline | 6 / 3 |
| 32768 / 256 | Baseline / candidate / baseline | 6 / 3 |

The 4K percentage averages the five pair-mean ratios. The 8K/32K percentage averages the three request-wise ratios against the matching before/after baseline means.

</details>

<details>
<summary>Baseline variability and actual speculative work</summary>

**4K**

Pair changes, TTFT: -0.234%, -0.197%, -0.299%, -0.203%, -0.281%; TPOT: -6.087%, -6.131%, -6.037%, -6.054%, -6.169%.
Baseline first-to-last change: TTFT +0.014%, TPOT +0.006%; block-mean spans 0.039% / 0.028%.
Output token IDs match across all arms, including warmups. Mean windows/offered/accepted per measured request: baseline 54.000/111.000/74.000 (15 requests); candidate 54.000/111.000/74.000 (15 requests).

**8K**

Baseline drift: TTFT +0.024%, TPOT -0.011%.
Output token IDs match across all arms, including warmups. Mean windows/offered/accepted per measured request: baseline 102.667/209.000/153.667 (6 requests); candidate 102.667/209.000/153.667 (3 requests).

**32K**

Baseline drift: TTFT +0.007%, TPOT +0.029%.
Output token IDs match across all arms, including warmups. Mean windows/offered/accepted per measured request: baseline 103.333/202.333/154.000 (6 requests); candidate 103.333/201.667/154.000 (3 requests).

Speculation settings are fixed; offered counts include suffix drafts and actual work may differ because suffix selection is time-adaptive. The measurements are warmed and post-readiness; the host page cache was not flushed, so no disk-cold TTFT result is claimed.

</details>

## Limits

Changes are confined to the SYCL implementation files listed above; no test-source changes are included. The public documentation diff is empty. Other GPUs, Windows and multi-GPU operation were not validated; automatic selection uses upstream Battlemage classification with the shape/configuration guards described above. Runtime and performance validation remain limited to B70.