Issues / #2847
#2847 Mac mini M6 (16 GB): Q2_0 streaming works - measured 11.7->27.2 tok/s + arena fix for sub-32 GB Macs
open · @hansong80 · 0 comments · View on GitHub
BenchmarksSetup & installModels & quants
Description
First data point for a **sub-32 GB Mac**, following up on MAC.md's "smaller Macs are untested". **Machine:** Mac mini, Apple M6, 16 GB unified memory, macOS 27.0.1, ~185 GB free disk. **Setup:** `git clone` + `HF_ENDPOINT=https://hf-mirror.com ./setup.sh --yes` (mainland China network — direct huggingface.co connected but transferred 0 bytes; the mirror fixed it at ~70 MB/s). Selected pack: **Q2_0**, SSD expert streaming, MTP auto-off, context 32768. Engine built from source with the stock `./setup.sh` (Metal). ### Result The server starts and serves, but **every inference fails** on a 16 GB Mac out of the box: ``` [strata] the engine reported an error: metal prompt: no arena region for the prompt buffers ``` Cause: at the moment the arena is sized, `vm_stat` shows only ~200 MB free, and the default "keep 7 GB for macOS" rule (MAC.md) leaves nothing for the prompt buffers. ### Fix In `strata-q2_0.json` -> `env`: ```json "STRATA_METAL_KEEP_FREE_GB": "3", "STRATA_METAL_ARENA_GB": "8" ``` Restart with `./run-q2_0.sh`. Everything then works. ### Measured (engine's own "done:" lines, Q2_0, 32K ctx) | Condition | tok/s | |---|---:| | Cold cache (first request after start) | 11.7 | | Warmed up (subsequent short requests) | 14.6 – 27.2 | So a 16 GB M6 is usable — close to the 32 GB M4 mini's documented ~14 tok/s. ### Minor note `"enable_thinking": false` in the request is ignored (reasoning_content still generated); `reasoning_budget_tokens` is the working knob, as the server log helpfully says. Could MAC.md mention the KEEP_FREE_GB/ARENA_GB combination as the first thing to try below 32 GB? That would have saved me the debugging.
Related on this site
Quick links to related pages.