Issues / #1968
#1968 Proposal: opt-in macOS external/backend seam (strata-mlx + TinyGPU/Macuda evidence)
open · @rajpaulsingh6-stack · 0 comments · View on GitHub
BenchmarksSetup & installAMD / HIPNVIDIA / CUDAModels & quantsDocumentationWindowsLinux
Description
This is deliberately narrower than #836. I am **not** asking Strata to add a Metal backend or claim general macOS support. On 2026-10-11 I validated two separate experimental lanes on an M2 Max Mac Studio (32 GB) with an RTX 5060 Ti 16 GB in a Razer Core X V2 over USB4: 1. **TinyGPU + a patched Macuda userspace CUDA/runtime stack:** the GB206 card completed a value-checked VRAM round trip, ran an `sm_120` kernel, passed 450/450 ggml CUDA backend operations, and ran llama.cpp inference. Qwen3 0.6B Q4_K_M measured 22,597.26 +/- 205.96 tok/s prompt processing (1,024 tokens) and 393.80 +/- 1.10 tok/s generation (256 tokens). 2. **`yibie/strata-mlx`:** a pinned unofficial Swift 1.5 Qwen3.8 Flash-Next abliterated Q2_0 model ran with a 12 GB expert-memory cap, host suffix drafting, 8K context and MTP disabled. Its model pin is `SC117/Swift-1.5-Qwen3.8-Flash-Next-GSQ-RCO-abliterated-GGUF` at revision `2b89cf3f7f2d9cb84275439e2c0cda83db15d32f`; both GGUF shard hashes were verified. The `strata-mlx` lane completed two end-user acceptance tests: - It generated a constrained style/layout contract that GenOffice compiled into a four-sheet October 2026 NYC-area sports workbook with 88 team-game rows. The final workbook passed with 0 errors and 0 warnings. - It generated a 329-word Knicks narrative from three official NBA pages. After tightening citation and venue-direction gates, the final tuned run used 1,390 prompt + 579 completion tokens in 120.248 seconds, with one bounded retry and 0 final validation errors. Important boundary: these are **not one fused runtime**. Macuda proves the NVIDIA eGPU CUDA lane; `strata-mlx` proves that the target model and CPU/RAM expert-paging workflow are usable on this Mac. Current `Niko1221/Strata` mainline was not run on macOS, and TensorFold's macOS/Metal model footprint was not viable on 32 GB. Would you accept a small preparatory PR that adds an opt-in provider seam without changing the supported-platform promise? My proposed first commit is: 1. Introduce an explicit engine/provider descriptor for setup-time capability discovery and launch (`cuda`, `hip`, `sycl`, and an experimental external provider), instead of adding more direct OS/backend branches to `setup.py`. 2. Keep the external provider disabled unless the user supplies an explicit flag and command/config; Windows/Linux behavior remains unchanged. 3. Add card-free tests for provider selection, environment/config propagation, health/model-identity verification, and unsupported-host refusal. 4. Add an experimental hardware receipt schema (OS/arch, PCI ID, compute arch, VRAM, link width/rate, correctness checks, model hashes, prefill/decode). 5. Use that seam first for community experiments. Macuda and `strata-mlx` adapters would remain separately gated until each satisfies restart, sustained-load, correctness and failure-recovery tests. If this direction fits the project, I can submit the seam/tests first and keep all macOS-specific support claims out of the normal install path. References: - #836 (closed broad Apple-Silicon request) - [`docs/INSTALL.md`](https://github.com/Niko1221/Strata/blob/main/docs/INSTALL.md) - [`yibie/strata-mlx`](https://github.com/yibie/strata-mlx) - [`Davinchy/macuda`](https://github.com/Davinchy/macuda) - [Pinned Swift 1.5 Q2_0 model](https://huggingface.co/SC117/Swift-1.5-Qwen3.8-Flash-Next-GSQ-RCO-abliterated-GGUF)
Related on strata.com
Editorial links to help you install, pick models, or read release notes — not part of the upstream thread.