Pull requests / #3122

#3122 Metal backend: fix the build on macOS 27.2 / Apple clang 21 and make fused_gr_persist_init a no-op

open · @mnott · 0 コメント · GitHub で見る

BenchmarksSetup & installServer & APIAMD / HIPNVIDIA / CUDAModels & quants

本文

## Problem

Building the Metal engine of v0.1.45 (`./setup.sh`, Metal path) on macOS 27.2 with Apple clang 21 fails, and once it builds,
serving needs a workaround.

1. `metal/include/metal_compat/cuda_runtime.h` lacks `cudaDevAttrComputeCapabilityMajor/Minor`, `cudaDeviceGetStreamPriorityRange`,
   `cudaStreamCreateWithPriority`, `cudaMemcpyPeer` and `cudaDeviceDisablePeerAccess`, which `src/core/verify.cpp` and
   `src/core/peer_experts.cpp` use (the HIP compat header maps them).
2. `metal/src/engine/stubs.cpp` is stale: `Gemm::bf16` lacks the `bool x_f16` parameter (compile error), and 21 kernel entry
   points (`wait_handoff`, `hc_q8_to_bf16`, `qsa_*`, `gr_norm*`, ...) are undefined at link time.
3. The serve path calls exactly one generated stub at start, `strata::kernels::fused_gr_persist_init` (`src/core/verify.cpp`, the
   opt-in HIP initialiser behind `STRATA_HC_PERSIST=1`). Generated stubs stop the engine (exit 86) unless
   `STRATA_METAL_STUBS=log` is set, so the server could only be started with stub logging on.

## Change

- `cuda_runtime.h`: shims for the missing calls. Compute capability and the peer calls report "unsupported" (the only
  compute-capability caller, `src/core/verify.cpp`, reads a failure as "not a Turing card"); stream priority range 0/0, a
  priority stream is a plain stream.
- `stubs.cpp`: regenerated in place with `metal/tools/gen_stubs.py` against the v0.1.45 headers (its include root is the LAST
  `-I.../include` that contains `strata/`, so `-I<repo>/include` must come after `-I<repo>/metal/include`): stale signatures
  updated (`Gemm::bf16` and the `gr_norm*` family gained their `bool` parameter) and the 21 missing entry points added. They are
  not ports: they stop the engine with the entry point's name if reached, like the existing stubs.
- `setup_host.cpp`: `fused_gr_persist_init() {}` next to `fused_gr_check() {}`, and `fused_gr_read_persist()` returning `false`,
  both removed from the generated stubs. This matches the CUDA and SYCL builds (`src/kernels/cuda/fused_gr.cu`, non-HIP branch).
  All other stubs still stop the engine.

The build-fix parts overlap with #3114, which bundles them with RoPE scaling, a memory probe and setup changes; this PR is
only the build fix, tested on hardware, and adopts #3114's in-place stub regeneration and "unsupported" compute capability.

## Testing

Apple M5 Max, 128 GB, macOS 27.2, Apple clang 21.

- Build: `./setup.sh --yes --no-start --no-browser --family unsloth --model UD-Q4_K_XL --vision no --build` finishes and the
  engine links (exit 0).
- Serve without `STRATA_METAL_STUBS` (IQ3_S pack, 65,536 context, MTP on): the engine reaches READY and answers a short
  `/v1/messages` request and a tool-call request (`tool_use` with the right arguments, `stop_reason` `tool_use`). The engine log
  of that start has no `STUB` line, and the engine stayed up (no exit 86) until stopped. Before the change the same start logged
  `STUB strata::kernels::fused_gr_persist_init`.
- Speed, same machine: a 5-hour agentic coding run (Flash-Next IQ3_S, MTP on, thinking off, 262,144 context) served 2,843
  `/v1/messages` requests, measured from the engine's per-request log lines (prefill = newly read tokens / prefill time,
  decode = generated tokens / generation time):

| Context per request | Requests | Decode tok/s | Prefill tok/s |
|---|---|---|---|
| < 8k | 121 | 111.0 | 1352 |
| 8-45k | 1816 | 109.2 | 1398 |
| 45-90k | 771 | 108.6 | 1158 |
| 90-130k | 135 | 109.6 | 994 |

  Prefill counts requests with at least 4k new tokens; in the 90-130k row no request had that many (the rest of the prompt came
  from the prefix cache), so that value is from the 21 requests with 1-3k new tokens. Overall: 738,060 tokens generated at
  109.1 tok/s, prompts 98.0 % reused from the prefix cache, median context 33,100 tokens (p90 75,506, max 130,137), about 260
  tokens generated per request. Decode stays flat up to 130k context.
- Prefix cache check (5 turns, ~20k prompt): cold 25.4 s, follow-up turns 2.1-3.0 s, 93-94 % of the prompt reused.
- `/v1/messages` tool smoke (Read, Edit, Bash, Edit-after-Read): all pass.

関連リンク

インストール・モデル・リリースへの站内リンク。