#3662 [Engine] support Qwen3.8-Flash-Next Coder (256 experts) on Apple Silicon Metal

本文

## Problem and result

Strata previously supported Qwen3.8-Flash-Next 512-expert models on Apple Silicon (Metal), but attempting to run the Qwen3.8-Flash-Next Coder variant (256 experts) failed at multiple levels:
1. Setup scripts and profile selectors rejected it ("Select one of the three Qwen3.8-Flash-Next sizes (Q2_0, IQ3_XXS, IQ3_S); coder is not one of them").
2. The Metal kernels (`router.metal`, `moe.metal`, `prompt.metal`) hardcoded 512 experts with 16-unroll routing and MoE dispatches, causing out-of-bounds indexing or shape mismatches when given a 256-expert model.
3. The C++ engine (`window.cpp`, `stream.cpp`, `stream.h`, `prompt_chain.inc`) hardcoded 512 expert slots for routing logits, buffers, and offsets.
4. On macOS 15 SDK, runtime shader compilation failed when prompt tokens triggered prefill because `#include <MetalPerformancePrimitives/MetalPerformancePrimitives.h>` was unconditionally included in `dense_mm.metal` without feature detection.

With this change:
- `coder` family and `IQ1_M` are supported on Apple Silicon (Metal).
- Shaders and engine dynamically handle 256 experts alongside existing 512-expert models (100% byte-identical default path for 512-expert models).
- macOS 15 without Metal 4 / MPP safely falls back to universal decode window prompt prefill.
- Verified end-to-end inference in CLI and HTTP server on Apple M2 Max (96 GB Unified Memory) with speculative decoding (MTP) yielding ~29-32 tok/s decode and ~42 tok/s prefill.

## Related discussion
none

## Approach

1. Shaders:
- In `router.metal`: added `n_expert == 256` routing branch (unroll 8) alongside 512 (unroll 16).
- In `moe.metal`: added `n_expert == 256` MoE calculation branch.
- In `prompt.metal`: added `route_topk_256` kernel specialization.
- In `dense_mm.metal` & `mm_bench.metal`: guarded `MetalPerformancePrimitives` include and `gemm_f16` with `#if __has_include(<MetalPerformancePrimitives/MetalPerformancePrimitives.h>)`.

2. C++ Engine:
- In `stream.h`, `stream.cpp`, `window.cpp`, `prompt_chain.inc`: replaced static 512 constants with `n_expert_` detected dynamically from model metadata.
- In `window.cpp`: added `supports_metal4()` check to fall back cleanly to decode window prefill on M1-M4 / macOS 15 devices.

3. Setup & Control:
- In `setup.py`, `metal/setup_mac.py`, `strata_control/mac_fit.py`: enabled `coder` family and `IQ1_M` on macOS, configured shard file naming and profile defaults.
- In `metal/tools/stream_experts.py`: read `n_expert` from GGUF metadata instead of assuming 512.

## Validation

**Passed:**
- macOS Setup test suite on feat/metal-coder:
  `python3 tools/test_setup_mac.py` -> Ran 24 tests in 0.090s, OK.
- Metal build:
  `ninja -C build-metal strata` -> clean build without errors.
- Direct CLI Inference (101 tokens prompt prefill + decode):
  `./build-metal/strata --tokens "100,101...200" --pack <pack> --native <shard1> --ple-gguf <shard2> --expert-profile data/expert-profile-coder.bin --expert-cache 64 --prefill auto --spec 2 --mtp <mtp_dir> --max-context 65536 --mmap-experts --max-new 5`
  Result:
  - Prefill 100 tokens in 2344.2 ms (42.66 tok/s).
  - Decode 5 tokens in 169.7 ms (29.46 tok/s).
  - Speculation: 3 rounds, drafts accepted 2 of 2 (1.000).
- HTTP Server multi-turn chat (`/v1/chat/completions`):
  - Request 1: 53 prompt tokens, 20 generated tokens at 29.4 tok/s.
  - Request 2 (follow-up): 95 prompt tokens (48 reused + 47 read), 50 generated tokens at 32.2 tok/s, draft acceptance 27 of 29 (93.1%).
- Hardware environment: Apple M2 Max (12 cores, 38-core GPU), 96 GB Unified Memory, macOS 15.7.5 (Build 24G624).

**Failed:**
none observed

**Not run:**
CUDA / ROCm / SYCL execution (change is isolated to Apple Metal backend and macOS setup scripts; non-Mac code paths unaffected).

## Limits
On Apple Silicon without Metal 4 (M1-M4, macOS 15), prompt prefill uses the universal decode window batched prefill path; Metal 4 matrix-unit prefill (`matmul2d`) requires macOS 26+ and supported hardware.