#3641 [Engine] opt-in Sage-A1.5 F16 inference with faster 256K decode

Beschreibung

Sage-A1.5's publisher F16 GGUF cannot load through the release qwen35moe path. This adds opt-in `--f16-weights` execution and explicit `--kv f16`. On an RTX 5090 32 GB / Ryzen 9 9950X3D / 128 GB RAM, the fresh near-full run generates **36.58 tok/s versus llama.cpp's 29.95 (+22.2%)**.

Both final runs use the same F16 weights and F16 KV, 262,144 context capacity, **261,888 actual input tokens plus 256 greedy outputs**, 512-token chunks and MTP off. The llama.cpp reference was tuned to 27 CPU-expert layers / 8 threads; Strata uses 15 pool workers plus caller / 60 resident experts per layer. There is one fresh run per final configuration; the report separates saved-prefix tuning repeats and retains the original losing F16 prototype (7.46 tok/s).

The implementation adds raw F16 dense/embedding/expert kernels, reuses F16 prompt GEMMs, expert streaming and ggml's CPU F16 path, and shares coalesced KV reads across each group of eight query heads. The opt-in hybrid cache limits prompt routing history and updates every layer's scores so it can adapt during generation. F16 KV/session copies and prefill reads use the correct element width. Omitting the new flags retains the release quantized/FP32-cache paths; model catalogs and setup are unchanged.

Related: #2464 and #3303. This is scoped to the measured Sage F16 path, so those broader reports remain open. FP32 stays the default; this introduces no int8 KV format or global precision change.

[Full report, raw measurements, commands, outputs and provenance](https://github.com/acrogenesis/Strata/blob/f3f2136f194bdc42d07961bef0ce66b2811f58cb/bench/results/2026-10-21-sage-f16-rtx5090/README.md).

**Passed**

- CUDA 12.9 / GCC 14.3.1, HIP gfx1201 and SYCL oneAPI 2026.1 full builds. SYCL still does not run this family; its build checks the shared code.
- CUDA CTest 4/4: model layout, independent F16 matrix/expert oracle, independent F32/F16 attention oracle including 262,144 cells/causal tails/session tables, and existing Q8 expert parity.
- Actual model: 128 teacher-forced outputs after 16K input, mean KL 0.001957 / 0.002022 in two runs, 126/128 matching top-token choices in each.
- Release v0.2.2 versus candidate: 16 synthetic actual-width Q8 cases, identical token ids and complete logit bytes, GPU/hybrid placements, streamed prompts and fixed/changing expert slots. Separate MTP 0/2 protocol runs match the release and plain decode. The published fixture generator reproduces the tested file's SHA256.
- Small F16 fixtures: MTP 0/2/7 and session/batch-copy checks with fixed slots.

**Preparation failures resolved:** the first 16K reference helper exceeded its batch; it now reads 512-token chunks. An older SYCL toolchain failed unchanged dependency helpers; the documented 2026.1 build passed. A synthetic-fixture SSM width was corrected and its 96 tensor shapes checked before GPU regression. Details are retained in the report.

**Not run / limits:** AMD hardware execution, Sage vision, multi-GPU, real-model MTP/concurrent serving, long-context task accuracy and other model quantizations. The default regression uses synthetic weights with the actual layer widths, not a downloaded quantized Sage checkpoint. Preparatory CPU work overlapped prefill; only the final generation windows are the acceptance comparison. No general hardware or prefill-speedup claim.

Draft for scope feedback and AMD runtime validation: an AMD owner can run `q36_f16_parity`, `q36_attn_parity` and a real F16 prompt/decode check. CUDA execution has been validated; HIP compilation alone does not establish AMD execution correctness.