Pull requests / #2366

#2366 bench: community report - Apple M1 Max, 32 GiB, Q2_0 streaming

open · @demetree · 0 Kommentare · Auf GitHub

BenchmarksSetup & installModels & quantsDocumentation

Beschreibung

Measured 2026-10-14 with tools/strata_bench.py (suite 1, three runs per test, medians with ranges), on an M1 Max with 32 GiB of RAM and the Q2_0 pack streamed from the SSD - the install's own configuration (--expert-cache 64 --prefill auto --spec 2 --mmap-experts, STRATA_METAL_STREAM=1):

- short chat: decode 10.6 tok/s (8.6-12.7), first token 4.09 s (3.39-4.50)
- 4K prompt, fresh: 223.7 tok/s (217.8-224.1), first token 18.03 s
- 16K prompt, fresh: 228.6 tok/s (224.6-230.0), first token 70.33 s
- agent session: 68.7 % cache hit, 5 re-read tokens a session, last-turn first token 18.1 s, decode 9.2 tok/s

Results only: the report folder (README.md, per-run results.json, the engine log excerpt); no engine change. Speed suite, so no correctness check; background load not recorded. The engine that measured it is 0.1.44.1-metal on commit bef1c42c.

## Title
<!-- If Applicable, reference the GitHub issue -->
Issue: Resolves # 

## Summary
<!-- Quick Summary of changes -->

## What changed
<!-- Specifics on files changed, and what changes were made there -->

## Extra Notes
<!-- Any extra notes, delete if there are none -->

Mehr auf der Site

Links zu Install, Modellen, Releases.