Pull requests / #3067

#3067 Community benchmark: Apple M1 Max 64GB (Metal) - Q2_0, IQ3_XXS, IQ3_S

open · @henryhfhe · 0 コメント · GitHub で見る

BenchmarksModels & quantsDocumentation

本文

Results-only submission following docs/COMMUNITY_BENCHMARKS.md. Hardware: 16-inch MacBook Pro, Apple M1 Max (32-core GPU), 64GB unified memory, macOS, Metal backend, Strata commit 38978a6d. Model: Qwen3.8-Flash-Next GSQ-RCO packs. Configs: Q2_0 and IQ3_XXS resident (STRATA_METAL_STREAM=0, --spec 2); IQ3_S streamed (arena 38GB, LEAVE_SHARE 0.3). Short-chat decode medians: Q2_0 35.1, IQ3_XXS 32.0, IQ3_S 23.1 tok/s; 16K prefill about 380 / 320 / 290 tok/s. Needle-in-a-haystack 6/6 found at 16K and 30K (depths 10/50/90) for all three. Limitations: context configured at 32768 so the 32K speed test and 128K needle were not run; speed suite and needle only, no quality eval. First Apple Silicon report in bench/results.

関連リンク

インストール・モデル・リリースへの站内リンク。