Pull requests / #3650
#3650 [Benchmark] community report: Apple M1 Max, 32 GiB RAM, Q2_0 streaming
open · @demetree · 0 commentaires · Sur GitHub
BenchmarksSetup & installModels & quantsDocumentation
Description
## Title [Benchmark] community report: Apple M1 Max, 32 GiB RAM, Q2_0 streaming Issue: Resolves # ## Problem and result There was no Apple Silicon community report in `bench/results/`. An M1 Max with 32 GiB of RAM cannot hold the 46 GB Q2_0 pack, so it streams the experts from the SSD — a configuration worth recording, since the existing Mac coverage is an M4 mini (32 GB, streaming) and an M5 Max (128 GB, resident). Measured **2026-10-21 on 0.2.2 / engine 0.1.47-metal** with `tools/strata_bench.py` (suite 1, three runs per test, medians with ranges, 772 s total), install configuration unchanged: | Test | Result | |---|---| | short chat decode | 11.8 tok/s (9.8–14.1, n=9); first token 3.87 s (3.39–4.18) | | 4K prompt, fresh (4,019 tok) | 222.7 tok/s (222.0–227.3); first token 18.12 s | | 16K prompt, fresh (16,058 tok) | 227.1 tok/s (220.7–227.4); first token 70.78 s | | agent session | 69.5 % cache hit; 5 re-read tokens per session; last-turn first token 18.33 s; decode 9.7 tok/s | For context, the same box, pack and suite measured on 0.1.44.1 a week earlier (2026-10-14): decode 10.6 tok/s, 4K 223.7 tok/s, 16K 228.6 tok/s. Decode is up about 11 % on 0.2.2; the prompt numbers are inside the run-to-run range. ## Approach The folder the benchmark wrote, added under `bench/results/` as `docs/COMMUNITY_BENCHMARKS.md` asks: `README.md`, per-run `results.json`, and the engine's start log (`engine-log-excerpt.txt`). Results only — no engine change — so a reviewer can assess the measurements directly. (`strata bench run` is the standard protocol; this is the quick `bench.sh` suite, which the guide keeps valid.) ## Validation - **Passed**: every test completed its three runs — three fresh 4K prompts, three fresh 16K prompts, three agent sessions, nine short chats. The server ran the install's own config: Q2_0, `--expert-cache 64 --prefill auto --spec 2 --max-context 32768 --mmap-experts`, `STRATA_METAL_STREAM=1`. - **Failed**: nothing. - **Not run**: correctness/needle checks (speed suite only); thermal, clock and background-load capture. ## Limits - One machine, one configuration. The fresh-prompt tests defeat the conversation cache by design, so those are cold-prompt numbers; the agent turns reuse the model's own earlier replies, so token counts differ slightly between runs (the ranges show it). - The measured tree is `main` at `011c9e60` (0.2.2) plus this report branch; the report's commit line is the report branch itself. - A resident-pack M1 Max would not look like this, and these numbers do not carry over to a resident configuration. - Home folders, user and host names and addresses were replaced by the tool; only the loopback URL `127.0.0.1` remains in the engine log excerpt. I re-read all three files before submitting.
Sur le site
Liens install, modèles, releases.