Pull requests / #2506

#2506 Community benchmark: Mac Studio M5 Max 64 GB, IQ3_XXS and UD-Q4_K_XL streamed from the SSD

open · @joesinvestments · 0 コメント · GitHub で見る

BenchmarksSetup & installModels & quantsDocumentation

本文

## Title
Issue: none (a community report, per docs/COMMUNITY_BENCHMARKS.md). Related: #2502, #2504.

## Summary
A report from a Mac the maintainers do not have: M5 Max (40-core GPU) with 64 GB, where both IQ3_XXS and UD-Q4_K_XL stream
their experts from the SSD. `tools/strata_bench.py` on the v0.1.44 Metal engine, four runs on the same day: the setup's
configuration and a tuned one for each pack. The tuned settings use existing options only (MTP with a Q8_0 draft layer from
`tools/mtp_pack.py`, `STRATA_METAL_ARENA_GB`, `STRATA_METAL_PROMPT_CHUNK`, `STRATA_ROUTE_TAIL_SKIP=0`).

| | IQ3_XXS setup | IQ3_XXS tuned | UD-Q4_K_XL setup | UD-Q4_K_XL tuned |
|---|---:|---:|---:|---:|
| short chat, decode tok/s | 45.4 | 48.7 | 29.7 | 42.8 |
| 16K prompt, prompt tok/s | 1434.3 | 1663.6 | 700.8 | 1049.6 |
| agent session, decode tok/s | 42.4 | 59.9 | 26.4 | 44.8 |
| agent session, first token (s) | 2.76 | 2.06 | 10.55 | 5.19 |

## What changed
- `bench/results/2026-10-14-community-m5-max-64gb/`: a summary `README.md` (hardware, the same-day table, how the tuned
  settings were chosen with the measured sweep behind each, what was not tested) and the tool's own four reports.
- `docs/COMMUNITY_BENCHMARKS.md`: one entry in the list of community reports. Nothing else.

## Extra Notes
- The setup turns MTP off for a streaming Mac below 96 GB. On this 64 GB Mac `--spec 3` / `--spec 4` with a Q8_0 draft layer
  was faster, with byte-identical greedy output in every run we hashed. Offered as data, not as a request to change a
  default; it is one machine.
- `tools/strata_bench.py` labels the machine "CPU only, VRAM not read" on a Mac (it does not read Apple GPUs); the summary
  says what the hardware is.
- Privacy: the tool's placeholders cover home folders, user and host names; we also checked the files by hand.

関連リンク

インストール・モデル・リリースへの站内リンク。