Issues / #1968

#1968 Proposal: opt-in macOS external/backend seam (strata-mlx + TinyGPU/Macuda evidence)

open · @rajpaulsingh6-stack · 0 comentarios · En GitHub

BenchmarksSetup & installAMD / HIPNVIDIA / CUDAModels & quantsDocumentationWindowsLinux

Descripción

This is deliberately narrower than #836. I am **not** asking Strata to add a
Metal backend or claim general macOS support.

On 2026-10-11 I validated two separate experimental lanes on an M2 Max Mac
Studio (32 GB) with an RTX 5060 Ti 16 GB in a Razer Core X V2 over USB4:

1. **TinyGPU + a patched Macuda userspace CUDA/runtime stack:** the GB206 card
   completed a value-checked VRAM round trip, ran an `sm_120` kernel, passed
   450/450 ggml CUDA backend operations, and ran llama.cpp inference. Qwen3
   0.6B Q4_K_M measured 22,597.26 +/- 205.96 tok/s prompt processing (1,024
   tokens) and 393.80 +/- 1.10 tok/s generation (256 tokens).
2. **`yibie/strata-mlx`:** a pinned unofficial Swift 1.5 Qwen3.8 Flash-Next
   abliterated Q2_0 model ran with a 12 GB expert-memory cap, host suffix
   drafting, 8K context and MTP disabled. Its model pin is
   `SC117/Swift-1.5-Qwen3.8-Flash-Next-GSQ-RCO-abliterated-GGUF` at revision
   `2b89cf3f7f2d9cb84275439e2c0cda83db15d32f`; both GGUF shard hashes were
   verified.

The `strata-mlx` lane completed two end-user acceptance tests:

- It generated a constrained style/layout contract that GenOffice compiled
  into a four-sheet October 2026 NYC-area sports workbook with 88 team-game
  rows. The final workbook passed with 0 errors and 0 warnings.
- It generated a 329-word Knicks narrative from three official NBA pages.
  After tightening citation and venue-direction gates, the final tuned run used
  1,390 prompt + 579 completion tokens in 120.248 seconds, with one bounded
  retry and 0 final validation errors.

Important boundary: these are **not one fused runtime**. Macuda proves the
NVIDIA eGPU CUDA lane; `strata-mlx` proves that the target model and CPU/RAM
expert-paging workflow are usable on this Mac. Current `Niko1221/Strata`
mainline was not run on macOS, and TensorFold's macOS/Metal model footprint was
not viable on 32 GB.

Would you accept a small preparatory PR that adds an opt-in provider seam
without changing the supported-platform promise? My proposed first commit is:

1. Introduce an explicit engine/provider descriptor for setup-time capability
   discovery and launch (`cuda`, `hip`, `sycl`, and an experimental external
   provider), instead of adding more direct OS/backend branches to `setup.py`.
2. Keep the external provider disabled unless the user supplies an explicit
   flag and command/config; Windows/Linux behavior remains unchanged.
3. Add card-free tests for provider selection, environment/config propagation,
   health/model-identity verification, and unsupported-host refusal.
4. Add an experimental hardware receipt schema (OS/arch, PCI ID, compute arch,
   VRAM, link width/rate, correctness checks, model hashes, prefill/decode).
5. Use that seam first for community experiments. Macuda and `strata-mlx`
   adapters would remain separately gated until each satisfies restart,
   sustained-load, correctness and failure-recovery tests.

If this direction fits the project, I can submit the seam/tests first and keep
all macOS-specific support claims out of the normal install path.

References:

- #836 (closed broad Apple-Silicon request)
- [`docs/INSTALL.md`](https://github.com/Niko1221/Strata/blob/main/docs/INSTALL.md)
- [`yibie/strata-mlx`](https://github.com/yibie/strata-mlx)
- [`Davinchy/macuda`](https://github.com/Davinchy/macuda)
- [Pinned Swift 1.5 Q2_0 model](https://huggingface.co/SC117/Swift-1.5-Qwen3.8-Flash-Next-GSQ-RCO-abliterated-GGUF)

En el sitio

Enlaces a install, modelos, releases.