Issues / #2762
#2762 [Feature]: Add opt-in Muse Spark Open 816B-A42B support
open · @BColsey · 1 comentários · No GitHub
Setup & installNVIDIA / CUDAModels & quantsLinux
Descrição
### What do you want to do? Run **Meta Muse Spark Open 816B-A42B** through Strata's local inference engine, with expert offloading rather than requiring the whole checkpoint in GPU memory. Model: https://huggingface.co/meta-models/Muse-Spark-Open-816B-A42B Available GGUF conversions: https://huggingface.co/unsloth/Muse-Spark-Open-816B-A42B-GGUF This request is specifically for the newly released **Spark Open**, not Muse Glimmer 30B. Spark Open has 816B total / 42B active parameters, gated grouped-query attention, and 256 routed experts with top-8 routing. It is a different architecture from Qwen3.8 Flash Next; accepting its model name or repacking tensors alone would not provide inference support. ### Initial scope - Opt-in **text inference** for Muse Spark Open, preserving existing Qwen defaults and behavior. - Validate model metadata, tensor shapes, quantization support and memory requirements before allocating or downloading large files. - Reuse Strata's expert offloading where compatible, with explicit errors for unsupported features or formats. - Correct tokenizer/chat-template handling and API generation, with regression coverage. - Deterministic small-shape reference/parity tests, followed by CUDA validation and clearly separated full-checkpoint evidence. Vision/audio, performance claims and automatic installation as a recommended model are not part of the initial text-only implementation. ### Hardware available for testing Linux x86-64, Ryzen 9 3900X, 64 GB installed system RAM (about 60 GiB usable), RTX 3080 10 GiB + RTX 2080 Ti 11 GiB + RTX 3060 12 GiB. These cards have separate allocations, not a single 33 GiB device. Existing Qwen service must be preserved. The smallest currently published Unsloth GGUF is approximately 176 GB, so full-checkpoint execution on this hardware would require storage-backed offloading; it is **not yet demonstrated to fit or run at usable speed**. Small CUDA tests are not evidence of successful 816B inference. I am investigating an implementation and will link a PR with the exact validation performed and any outstanding limitations. AI-assisted development and independent review will be disclosed in that PR.
No site
Links install, modelos, releases.