Pull requests / #3299

#3299 Metal: run Gyro (trellis TQ2_T/TQK6/TQK7) models on Apple Silicon

open · @mnott · 0 comentarios · En GitHub

needs-format

Descripción

Gyro-M now runs on the Metal engine. Until now Metal refused Gyro files at load ("no trellis kernels").

**This PR builds on #3231** (Mac conversation cache). Its first 4 commits are that PR; the 5 Gyro commits follow.

## What's in it

- **Kernels.** TQ2_T (the type Gyro-M uses for gate, up and down), TQK6 and TQK7 (Gyro-S): decode GEMV, the v2 prompt expert GEMMs, and the Hadamard rotation. The CPU reference is `src/kernels/cpu/tq_cpu.cpp`.
- **Engine.**
  - `native_expert_supported` accepts types 144-146.
  - The Hadamard rotations are wired into the decode window and the prompt chain.
  - The trellis gate/up tile uses 16 lanes, because 80 units per row do not split evenly across 32.
- **Single-GGUF Gyro.** The PLE table (`per_layer_token_embd`, ~54 GB, contiguous) sits inside the model file. It is now excluded from the wrapped GPU buffers: only the head and tail ranges are wrapped (37.6 of 92 GB), with no read-ahead on the table. The table is still read with direct I/O. Files with a separate PLE shard (IQ3_S) keep the old path; `STRATA_METAL_WRAP_PLE=1` forces it.
- **Tests.** `metal/tests/tq_parity`: 81/81 PASS (rel L2 ~1e-7 against the exact dot; the Hadamard rotation is bit-identical to `hadamard_rotate`). `moe_mm_parity` PASS. `docs/GYRO.md` updated.

## Measured (M5 Max, 128 GB)

| | IQ3_S | Gyro-M |
|---|---|---|
| Engine footprint | ~61-65 GB RSS | 23-26 GB + 37.6 GB weights mapped |
| 20k prompt decode, prose / edit | 84 / 121 tok/s | 73-79 / 110 tok/s |
| Per-layer expert GEMV (top-10) | 57.6 us, 405 GB/s | 65-67 us, 190-194 GB/s |
| Agent coding task (3 parts), one agent | 7.4 min | 8.1 min |
| Same task, quality gates (tsc, build, full tests) | all pass | all pass |

- **Two engines on one Mac.** Because the mapped weights are shared, two Gyro-M engines fit in 128 GB (2 x 24 GB + 37.6 GB). Running one agent each did 1.28x the work of a single engine; per-engine decode drops to 56-69 tok/s on the shared GPU.
- **Where decode could get faster.** It is ALU-bound on the trellis hash/lookup; an int8 LUT is the obvious next step.

## Not covered

- TQK6/TQK7 have not run on a full model (no Gyro-S file).
- `STRATA_METAL_STREAM=1` is encoded for trellis but untested.
- The kernel source path is compiled in; `STRATA_METAL_KERNELS` overrides it.

En el sitio

Enlaces a install, modelos, releases.