Issues / #2762

#2762 [Feature]: Add opt-in Muse Spark Open 816B-A42B support

open · @BColsey · 1 comentários · No GitHub

Setup & installNVIDIA / CUDAModels & quantsLinux

Descrição

### What do you want to do?

Run **Meta Muse Spark Open 816B-A42B** through Strata's local inference engine, with expert offloading rather than requiring the whole checkpoint in GPU memory.

Model: https://huggingface.co/meta-models/Muse-Spark-Open-816B-A42B
Available GGUF conversions: https://huggingface.co/unsloth/Muse-Spark-Open-816B-A42B-GGUF

This request is specifically for the newly released **Spark Open**, not Muse Glimmer 30B. Spark Open has 816B total / 42B active parameters, gated grouped-query attention, and 256 routed experts with top-8 routing. It is a different architecture from Qwen3.8 Flash Next; accepting its model name or repacking tensors alone would not provide inference support.

### Initial scope

- Opt-in **text inference** for Muse Spark Open, preserving existing Qwen defaults and behavior.
- Validate model metadata, tensor shapes, quantization support and memory requirements before allocating or downloading large files.
- Reuse Strata's expert offloading where compatible, with explicit errors for unsupported features or formats.
- Correct tokenizer/chat-template handling and API generation, with regression coverage.
- Deterministic small-shape reference/parity tests, followed by CUDA validation and clearly separated full-checkpoint evidence.

Vision/audio, performance claims and automatic installation as a recommended model are not part of the initial text-only implementation.

### Hardware available for testing

Linux x86-64, Ryzen 9 3900X, 64 GB installed system RAM (about 60 GiB usable), RTX 3080 10 GiB + RTX 2080 Ti 11 GiB + RTX 3060 12 GiB. These cards have separate allocations, not a single 33 GiB device. Existing Qwen service must be preserved.

The smallest currently published Unsloth GGUF is approximately 176 GB, so full-checkpoint execution on this hardware would require storage-backed offloading; it is **not yet demonstrated to fit or run at usable speed**. Small CUDA tests are not evidence of successful 816B inference.

I am investigating an implementation and will link a PR with the exact validation performed and any outstanding limitations. AI-assisted development and independent review will be disclosed in that PR.

No site

Links install, modelos, releases.