Issues / #2847

#2847 Mac mini M6 (16 GB): Q2_0 streaming works - measured 11.7->27.2 tok/s + arena fix for sub-32 GB Macs

open · @hansong80 · 0 comentários · No GitHub

BenchmarksSetup & installModels & quants

Descrição

First data point for a **sub-32 GB Mac**, following up on MAC.md's "smaller Macs are untested".

**Machine:** Mac mini, Apple M6, 16 GB unified memory, macOS 27.0.1, ~185 GB free disk.
**Setup:** `git clone` + `HF_ENDPOINT=https://hf-mirror.com ./setup.sh --yes` (mainland China network — direct huggingface.co connected but transferred 0 bytes; the mirror fixed it at ~70 MB/s). Selected pack: **Q2_0**, SSD expert streaming, MTP auto-off, context 32768. Engine built from source with the stock `./setup.sh` (Metal).

### Result

The server starts and serves, but **every inference fails** on a 16 GB Mac out of the box:

```
[strata] the engine reported an error: metal prompt: no arena region for the prompt buffers
```

Cause: at the moment the arena is sized, `vm_stat` shows only ~200 MB free, and the default "keep 7 GB for macOS" rule (MAC.md) leaves nothing for the prompt buffers.

### Fix

In `strata-q2_0.json` -> `env`:

```json
"STRATA_METAL_KEEP_FREE_GB": "3",
"STRATA_METAL_ARENA_GB": "8"
```

Restart with `./run-q2_0.sh`. Everything then works.

### Measured (engine's own "done:" lines, Q2_0, 32K ctx)

| Condition | tok/s |
|---|---:|
| Cold cache (first request after start) | 11.7 |
| Warmed up (subsequent short requests) | 14.6 – 27.2 |

So a 16 GB M6 is usable — close to the 32 GB M4 mini's documented ~14 tok/s.

### Minor note

`"enable_thinking": false` in the request is ignored (reasoning_content still generated); `reasoning_budget_tokens` is the working knob, as the server log helpfully says. Could MAC.md mention the KEEP_FREE_GB/ARENA_GB combination as the first thing to try below 32 GB? That would have saved me the debugging.

No site

Links install, modelos, releases.