改动 / #3650

#3650 [Benchmark] community report: Apple M1 Max, 32 GiB RAM, Q2_0 streaming

open · @demetree · 0 评论 · 去 GitHub 看

BenchmarksSetup & installModels & quantsDocumentation

说明

## Title
[Benchmark] community report: Apple M1 Max, 32 GiB RAM, Q2_0 streaming

Issue: Resolves #

## Problem and result
There was no Apple Silicon community report in `bench/results/`. An M1 Max with 32 GiB of RAM cannot
hold the 46 GB Q2_0 pack, so it streams the experts from the SSD — a configuration worth recording,
since the existing Mac coverage is an M4 mini (32 GB, streaming) and an M5 Max (128 GB, resident).

Measured **2026-10-21 on 0.2.2 / engine 0.1.47-metal** with `tools/strata_bench.py` (suite 1, three runs
per test, medians with ranges, 772 s total), install configuration unchanged:

| Test | Result |
|---|---|
| short chat decode | 11.8 tok/s (9.8–14.1, n=9); first token 3.87 s (3.39–4.18) |
| 4K prompt, fresh (4,019 tok) | 222.7 tok/s (222.0–227.3); first token 18.12 s |
| 16K prompt, fresh (16,058 tok) | 227.1 tok/s (220.7–227.4); first token 70.78 s |
| agent session | 69.5 % cache hit; 5 re-read tokens per session; last-turn first token 18.33 s; decode 9.7 tok/s |

For context, the same box, pack and suite measured on 0.1.44.1 a week earlier (2026-10-14): decode
10.6 tok/s, 4K 223.7 tok/s, 16K 228.6 tok/s. Decode is up about 11 % on 0.2.2; the prompt numbers are
inside the run-to-run range.

## Approach
The folder the benchmark wrote, added under `bench/results/` as `docs/COMMUNITY_BENCHMARKS.md` asks:
`README.md`, per-run `results.json`, and the engine's start log (`engine-log-excerpt.txt`). Results only —
no engine change — so a reviewer can assess the measurements directly. (`strata bench run` is the standard
protocol; this is the quick `bench.sh` suite, which the guide keeps valid.)

## Validation
- **Passed**: every test completed its three runs — three fresh 4K prompts, three fresh 16K prompts, three
  agent sessions, nine short chats. The server ran the install's own config: Q2_0, `--expert-cache 64
  --prefill auto --spec 2 --max-context 32768 --mmap-experts`, `STRATA_METAL_STREAM=1`.
- **Failed**: nothing.
- **Not run**: correctness/needle checks (speed suite only); thermal, clock and background-load capture.

## Limits
- One machine, one configuration. The fresh-prompt tests defeat the conversation cache by design, so those
  are cold-prompt numbers; the agent turns reuse the model's own earlier replies, so token counts differ
  slightly between runs (the ranges show it).
- The measured tree is `main` at `011c9e60` (0.2.2) plus this report branch; the report's commit line is
  the report branch itself.
- A resident-pack M1 Max would not look like this, and these numbers do not carry over to a resident
  configuration.
- Home folders, user and host names and addresses were replaced by the tool; only the loopback URL
  `127.0.0.1` remains in the engine log excerpt. I re-read all three files before submitting.

本站相关内容

相关页面的快捷入口。