Issues / #2504

#2504 Metal: an MTP draft layer with Q4_0 experts stops the engine at the first request ("no function moe_gu_2_32_1_4")

open · @joesinvestments · 0 comentários · No GitHub

Setup & installModels & quants

Descrição

### What happened

`tools/mtp_pack.py` offers `--experts q2_0 | q4_0 | q8_0` (its header names `mtp-q4_0.gguf` the acceptance comparison
arm). On the Metal engine a Q4_0 draft layer loads, then the engine exits with code 3 at the first request, on every pack
we tried (IQ3_XXS, IQ3_S, UD-Q4_K_XL). The Q2_0 layer that setup writes and a Q8_0 layer built the same way both work.

The log names the cause: the Metal library has no MoE gate/up kernel for that expert type and tile:
`mtl: no function moe_gu_2_32_1_4 Function moe_gu_2_32_1_4 was not found in the library`.

If Q4_0 is not meant for Metal, a refusal at load (as for Qwen3.6) would be friendlier than an exit at the first request.

Steps:
```
.venv/bin/python tools/mtp_pack.py --src <Strata-data>/mtp --experts q4_0 --out <Strata-data>/mtp/mtp-q4_0.gguf
STRATA_METAL_MTP_GGUF=<Strata-data>/mtp/mtp-q4_0.gguf  (config args: --mtp <Strata-data>/mtp/rt --spec 3)
```
Then any chat request. With `mtp-q8_0.gguf` the same config answers normally, and its output equals the no-MTP output
token for token on our three test prompts.

### Strata version, GPU, OS

Strata v0.1.44 (ed4773eb) built from source with `./setup.sh`, Metal engine. Mac Studio M5 Max (40-core GPU), 64 GB,
macOS 27.0.1. Packs streaming from the SSD (`STRATA_METAL_STREAM=1`).

### Engine log

```
strata metal: MTP draft layer from <Strata-data>/mtp/mtp-q4_0.gguf (1.60 GB, 9 norms folded), draft head over 106299 tokens
strata metal stream: resident footprint after the fill: the process holds 34.6 GB (36.3 GB resident), planned 35.0 GB, cap 35.0 GB
mtl: no function moe_gu_2_32_1_4 Function moe_gu_2_32_1_4 was not found in the library
[strata] the engine stopped unexpectedly (exit code 3).
```

No site

Links install, modelos, releases.