Pull requests / #3114

#3114 Optimize strata m5 max metal rope scaling 3378619349143257926

open · @WillkTech · 0 Kommentare · Auf GitHub

Setup & installModels & quantsDocumentation

Beschreibung

Optimizes Strata for Apple Silicon (M5 Max 128GB):
- Wires RoPE YaRN scaling into Metal decode window table (`cs_`) in `metal/src/engine/window.cpp` via header-only helper `metal/src/engine/rope_table.h`.
- Adds `<algorithm>` include in `metal/tests/rope_parity.cpp` testing unscaled, linear, and YaRN RoPE tables against float64 reference math.
- Implements macOS host memory probe in `src/core/conversation_memory.cpp` using `host_statistics64` with page calculation matching `arena_budget.h`.
- Adds Metal guardrails for unsupported explicit `--kv` formats (`q4_0`, `k8v4`) in `src/program/generate.cpp`.
- Updates `metal/setup_mac.py` context laddering (supporting 262144 and 524288) with automatic RoPE YaRN resolution.
- Reconciles `mac_readme.md` and `docs/MAC.md` configuration details.
- Verified with C++ parity tests (`rope_parity`, `conversation_memory_test`) and Python setup unit tests (`test_setup_mac.py`, `test_setup_rope.py`).

Mehr auf der Site

Links zu Install, Modellen, Releases.