Issues / #2898
#2898 [Feature]: Swift 1.5 on a Mac (opt-in --family swift, IQ3_S)
open · @oceanio · 0 コメント · GitHub で見る
BenchmarksSetup & installAMD / HIPNVIDIA / CUDAModels & quantsDocumentation
本文
### What do you want to do? Run Swift 1.5 on a Mac. docs/MAC.md says the Swift release is "not set up on Mac yet", and `metal/setup_mac.py` only takes `--family qwen|unsloth`. **What blocks it.** Swift's two shards cut through a layer: in IQ3_S, layer 14's gate is in shard 1, and its up and down are in shard 2. Its shard 1 also holds the n-gram table. The Metal engine maps one file (`window.cpp` wraps shard 1), so it cannot read Swift as it is. **What already works.** - `metal/tools/metal_repack.py` (written for Unsloth's 4-6 shards) works on Swift's 2 shards as they are. - `tools/iq_pack.py` already handles Swift's layout. It writes the 96 F32 routers as BF16, and all 96 are exact. - The Metal gemv has IQ4_NL and IQ4_XS, which Swift uses for `attn_output`, `ssm_out` and `ffn_down_shexp`. - The router read takes F32. - The engine needs no change. **What I propose (opt-in).** `./setup.sh --family swift` sets up Swift 1.5 IQ3_S: setup repacks it like Unsloth's files, and leaves images off. Without `--family swift` nothing changes, including the size menu, the original model and Unsloth's files. Only IQ3_S is offered, because it is the only size I could measure. I have this on a branch: https://github.com/oceanio/Strata/tree/feat/mac-swift - One commit on `metal/setup_mac.py`, `tools/test_setup_mac.py` (26 tests, 6 new) and `docs/MAC.md`. - It sits on #2895: main does not build on a Mac without it (#2860). **Measured** on an M5 Max (128 GB, macOS 26.7.1) with SC117's abliterated Swift 1.5 IQ3_S (`SC117/Swift-1.5-Qwen3.8-Flash-Next-GSQ-RCO-abliterated-GGUF`, the same file layout as UkisAI's release). The model was resident, with `--spec 2` and MTP. Engine 0.1.45 with #2895, installed by the branch's setup end to end (`--gguf-dir`). | | Swift 1.5 IQ3_S | Qwen3.8-Flash-Next IQ3_S (SC117), same Mac | |---|---|---| | Decode, 4 prompts up to 200 tokens, greedy | 93.1 / 97.2 / 110.4 / 104.6 tok/s | 94.3 / 95.1 / 109.2 / 108.5 tok/s | | 6.8K-token needle, 3 documents | 3 of 3 | 3 of 3 | | Prompt, 6.8K tokens (warm) | 1,636-1,731 tok/s | 1,651-1,767 tok/s | | 3 short reasoning questions, thinking on | 3 of 3 right | 3 of 3 right | **Not tested.** - UkisAI's own files: I only have SC117's. - A comparison with the CUDA or HIP engine on the same Swift file. I have no NVIDIA or AMD card. - The KL harness. - Contexts over 8K, streaming on a Mac under ~96 GB, a long server run, and sizes other than IQ3_S. **Questions.** 1. Would you take this as an opt-in setup change? 2. Could someone with a CUDA or HIP card give reference outputs for Swift IQ3_S, for example greedy text on a few prompts? I would then compare the Metal engine against them, the way docs/MAC.md does for the original model. I would open the PR once #2895 is in, so that it holds only this commit.
関連リンク
インストール・モデル・リリースへの站内リンク。