#3427 [Engine] Metal: the control vector (experimental speed projection) on the Mac engine

本文

## Problem and result

On a Mac, `--control-vector-scaled` loads nothing: the Metal engine answers with an empty control vector (`metal/src/engine/missing.cpp`), and `docs/MAC.md` lists the control vector as not ported. So the experimental speed projection (`data/experimental-speed-projection/`, docs/DETAILS.md) does nothing on Apple Silicon, although the flags are accepted.

With this change, the Metal engine applies the vector as the CUDA engine does: after each steered layer (4..44 for the ESP), every hyper-connection stream of every row becomes `h - s (h.v) v` (project) or `h + d` (add). It covers the decode window, the MTP verify window and both prompt paths. The log prints `control vector mode = project, dir = per-layer, layers 4..44 (41 steered)`, and the request field `"experimental_speed_projection": false` switches it off. With it off, the replies are token-for-token those of the engine started without a vector.

## Related discussion

none (a port of the existing CUDA/HIP feature to the Metal backend; no new flag, no change to a default)

## Approach

- `metal/src/engine/msl/cvec.metal`: one threadgroup per (stream, row). It applies the layer's pending FFN write `R += bo * 2 sigmoid(inj / hc)` with `gr_norm`'s own arithmetic, then the projection (or the addition). It is the same operation as `src/kernels/cuda/cvec.cu`.
- `metal/src/engine/window.cpp`, `prompt_chain.inc`: after a steered layer's MoE output, the window encodes this kernel, and the next layer's hyper-connection read runs with `apply = false` (the write is already in R). The last layer and the final `output_hc` read follow the same rule. Three paths are covered: `Window::run` (decode, verify windows, short prompts in 8-row windows), `pc_chunk` (the resident prompt chain) and `prompt_chunk` (the streaming prompt chunks).
- The per-request switch: the window caches its encoded command lists, so the switch state is part of the cache key. A request with the vector off encodes exactly the stock dispatches. No device flag is needed, and an off request costs nothing.
- `metal/src/engine/cvec.cpp`: `cvec_upload`, `cvec_set_enabled`, `cvec_enabled`, `cvec_replicate` and `cvec_tables` for Metal (unified memory: the tables stay in host memory, and the window copies the 480 KB direction table into a Metal buffer the first time the vector is on). Their four stubs are removed from `stubs.cpp`, and the empty `cvec()` from `missing.cpp`. `cvec_apply` and `gr_write_cvec_norm_rs` stay stubs: they are the CUDA engine's host-side hooks (layer.cpp, verify.cpp, prefill.cpp), which the Metal engine does not run.
- No shared source changes (`src/`, `include/`): CUDA, HIP and SYCL builds are not touched.

## Validation

Revision: `metal-control-vector` at c885c306 (main 9ae93f7b + this change). MacBook Pro, Apple M5 Pro, 64 GB, macOS 27. Built with `cmake -S metal -B build-metal -G Ninja -DCMAKE_BUILD_TYPE=Release -DMETAL_CPP_DIR=… -DSTRATA_GGML_DIR=…` (llama.cpp 3cf03257) and `ninja -C build-metal strata cvec_parity`.

**Passed:**
- Build: `strata`, `cvec_parity`, `gr_parity` link with no error; the warnings are in existing shared headers only.
- `build-metal/cvec_parity`: 27 of 27 checks, for 1, 3 and 8 rows. Project against a double-precision host reference (max error 5.0e-7; s = 2 reflects the component, s = 1 removes it). The pending write is **bitwise** `gr_norm`'s folded write (gr.metal with `apply`). Write then project: max error 4.8e-7. Add is `h + d` bitwise. Rows past the window are untouched.
- Engine A/B, Qwen3.8-Flash-Next Q2_0 in memory (`--spec 2 --mtp … --prefill auto --max-context 65536 --mmap-experts`, `STRATA_METAL_STREAM=0`, `STRATA_METAL_PROMPT_CHUNK=2048`), served by `serve/server.py`, greedy (`temperature 0`, `reasoning_effort none`, `max_tokens 256`). Five prompts: 4 short ones (24-42 tokens, 8-row windows) and one of 2,338 tokens (the resident prompt chain). Two repetitions.
  - Started with `--control-vector-scaled <Strata>/data/experimental-speed-projection/Qwen3.8-Flash-Next-experimental-speed-projection.gguf:1.0 --control-vector-layer-range 4 44 --cvec-mode project --cvec-dir per-layer`: the log prints `control vector mode = project, dir = per-layer, layers 4..44 (41 steered)`.
  - `"experimental_speed_projection": false` against the same build started without the vector: 8 of 8 replies identical, reasoning and text, character for character (one of them cut short at 160 tokens by an external memory watchdog, a prefix of the stock reply). The stock engine is identical between its two repetitions.
  - `"experimental_speed_projection": true`: 8 of 8 replies differ from stock. The two requests the stock model refused were answered: "How do I pick a pin tumbler lock…" in both repetitions, "Write a brutal, mean-spirited roast…" in the one repetition that ran (the second stopped before it).
  - Cost, kernel alone (`cvec_parity --bench`, the 41 dispatches a window adds, median of 20): 0.22 ms of GPU time per window for 1, 2, 3 or 8 rows; 38 ms for a 2,048-row prompt chunk.
  - Cost, decode: about 25 ms per verify window in all three runs (on, off, stock); the 0.22 ms is about 1% of it. The tokens/s of the replies (on 55.8-73.9, off 61.0-72.2, stock 61.2-73.7) do not resolve a 1% cost: off and stock run the same dispatches and still differ by up to 5.5% on one prompt (the code prompt, mean 69.1 vs 73.1 tok/s).
  - Cost, prompt: the 2,338-token prompt read in 2,277 / 2,282 / 2,326 ms with the vector on and 2,262 / 2,302 / 2,319 ms off (alternating, after one warm-up pair): 2,295 vs 2,294 ms. The first read of a run is slower (2.9-10.4 s, cold file cache after another model server ran); it is left out.

**Failed:**
- none observed

**Not run:**
- The streaming path (`prompt_chunk`, experts read from the SSD with `STRATA_METAL_STREAM=1`): the code is there, not run.
- `--cvec-mode add`, and `--cvec-dir single:L` in the engine (the add kernel is covered by `cvec_parity` only).
- A KL or perplexity measurement against stock on Metal (like bench/results/2026-09-27-esp/ on CUDA).
- The M5 Max, the M4 and other Macs; Unsloth's files and the other families.
- CUDA, HIP and SYCL builds: no file they compile is changed.

## Limits

- The Mac setup (`metal/setup_mac.py`, `strata launch`) does not offer the speed projection yet. It is turned on by passing the engine flags (the config's `args`), as above.
- The vector is copied into the window the first time it is on; a second `cvec_upload` in the same process (not done by the loader) would not be picked up.
- `cvec_tables` returns false on Metal (no device tables for another kernel to fuse); the CUDA prompt path's `STRATA_CVEC_FUSE` has no Metal counterpart, and none is needed: the prompt kernels read the vector through the same dispatch.