#6253 [Engine] macOS / Metal: open contributions, rebased on v0.2.16
closed · @mnott · 3 评论 · 去 GitHub 看
BenchmarksServer & APINVIDIA / CUDAModels & quants
说明
### Area [Engine] (speed, kernels, models, the server and API) ### What do you want to do? Run the Gyro models (M, L) on a Mac with Metal on v0.2.16. Measured on an Apple M5 Max with 128 GB of unified memory. Everything from the v0.2.13 overview (#6028) landed upstream in v0.2.16, thank you: - #6024 (Metal wraps a Gyro file without its PLE table): on v0.2.16 Gyro-M TQ2_0 wraps 37.57 GB as 2 separate ranges of 91.97 GB; greedy output identical across 5 runs. - #6025 (Metal kernels for the pair-trellis expert types TQP5/6/7 used by Gyro-L): `tqp_parity` passes on the real Gyro-L file (144 tensors) and `expert_parity` passes for TQP5/6/7 and TQ2_T. - #6026 (a published PLE-only GGUF passed to `--ple-gguf` is not read as a model shard). - #6032 (serve tool-call recovery): taken upstream as narrow repairs; the remaining ideas were declined and are not re-proposed. - #6027 (VisionArgs serve test): passes on v0.2.16 on macOS without the patch, so it is closed rather than rebased. Open from our side: nothing. A live serving check of Gyro-L on v0.2.16 is still to come; if it shows a problem it will be reported as its own issue with numbers. One observation to check, not a request yet: on Gyro-M (CLI, greedy, same prompt, warm page cache) decode measured 50 and 68 tok/s on v0.2.16 against 65 and 79 tok/s on our v0.2.13 build. Only a few runs; it will be re-measured on Gyro-L before anything is filed. ### What do you do today instead? Gyro-M runs on v0.2.16 as is; Gyro-L still serves from the v0.2.13 build until the live check.
本站相关内容
相关页面的快捷入口。