Pull requests / #2895

#2895 metal: the Mac engine builds and links again at 0.1.45 (#2860)

open · @oceanio · 0 comments · View on GitHub

BenchmarksServer & APIMulti-GPUAMD / HIPNVIDIA / CUDAModels & quants

Description

## Title
Issue: Resolves #2860

## Summary
The Metal engine does not build at 0.1.45 (477ac13): six CUDA calls are missing from metal_compat, and the link
misses 21 kernel entry points. This adds them, the way stubs_144.cpp did for 0.1.44. Only files under metal/
change, so the CUDA, HIP and SYCL builds are untouched.

## What changed
- `metal/include/metal_compat/cuda_runtime.h`: the compute-capability attributes, the stream-priority calls,
  `cudaMemcpyPeer` and `cudaDeviceDisablePeerAccess` answer "unsupported". verify.cpp's sm_75 test says no, the
  opt-in high-priority stream falls back to the default stream, and the peer-copy probe (#2666) keeps peer access off.
- `metal/src/engine/stubs_145.cpp` (new): 21 entry points.
  - Five answer "not available" through the caller's documented fallback: `fused_gr_persist_init` (no-op; the
    Verifier calls it on every start, so a stopping stub would stop every start), `fused_gr_read_persist` (false:
    `fused_gr_read_multi`), `qsa_decode_attn_split` (false: the plain attention), and the two `STRATA_F16_RANGE`
    reads.
  - The rest stop the engine with their name. None is reached with the Metal defaults: batch rows, the layer-split
    hand-off, the mapped plan ring, the sm_75 q8_1 interleave, `STRATA_HC_REQ8`, the FP16 prompt images and AVX2 rows.
- `metal/src/engine/stubs.cpp`: `Gemm::bf16`'s stub takes 0.1.45's `x_f16` argument (one line; the generated file
  otherwise unchanged).

## How it was checked
M5 Max, 128 GB, macOS 26.7.1, Command Line Tools 27.0 (Apple clang 21). Model: Qwen3.8-Flash-Next GSQ-RCO IQ3_S
(SC117's abliterated build), resident, `--spec 2` with MTP. Compared against v0.1.44.1 on the same Mac, the two
servers run alternately:

| | v0.1.44.1 | this PR |
|---|---|---|
| Greedy, 4 prompts x up to 200 tokens (story, Chinese, code, primes) | reference | identical text |
| Decode, tok/s | 96.7 / 95.1 / 108.6 / 108.1 | 94.3 / 95.1 / 109.2 / 108.5 |
| 6.8K-token needle, 3 documents | found | found, same answers |
| Prompt, uncached, tok/s | 1,667 / 1,630 | 1,651 / 1,767 |
| "has no Metal implementation" in the engine log | - | none |

The build adds no new warnings.

## Extra Notes
Not tested: vision, contexts over 8K, the SSD-streamed experts (a Mac under ~96 GB), a long server run, and
other Apple chips. `res_patch` stops if reached. It was not reached here, but I did not prove it cannot be.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Related on this site

Quick links to related pages.