#6522 [Engine] Ollama-converted GGUF fails at load: qwen35moe.attention.head_count_kv is a per-layer array
open · @jffhhlch · 3 Kommentare · Auf GitHub
Server & APINVIDIA / CUDAModels & quantsWindows
Beschreibung
Summary
A Qwen3.6-35B-A3B GGUF file produced by Ollama (a community abliterated fine-tune, huihui_ai/qwen3.6-abliterated:35b-a3b-q4_K) fails to load with:
代码块
strata-q36: qwen35moe.attention.head_count_kv is not a number
The Strata desktop UI recognizes the file as "the layout of qwen36 UD-Q4_K_XL", but the engine exits (exit code 1) while reading the model's weights, before serving.
Root cause (verified by parsing the GGUF header)
I parsed the GGUF v3 header directly. The key facts:
代码块
gguf v3, 1194 tensors, 57 kv
general.architecture string qwen35moe
general.file_type u32 15 (Q4_K_M)
general.parameter_count u64 35951822704
qwen35moe.attention.head_count u32 16
qwen35moe.attention.head_count_kv array array<u32>[40] =
[0,0,0,2] x 10 (per-layer!)
qwen35moe.context_length u32 262144
qwen35moe.embedding_length u32 2048
qwen35moe.expert_count u32 256
qwen35moe.expert_feed_forward_length u32 512
qwen35moe.expert_shared_feed_forward_length u32 512
qwen35moe.expert_used_count u32 8
qwen35moe.feed_forward_length u32 0 (MoE-only)
So qwen35moe.attention.head_count_kv is stored as a per-layer array<u32>[40] (llama.cpp's convention when layers have heterogeneous KV head counts, here the repeating pattern [0, 0, 0, 2]), while the qwen36 reader apparently expects a scalar u32 and crashes with "is not a number".
Note: llama.cpp itself accepts head_count_kv as either a scalar or a per-layer array, so GGUF files written by other converters may legitimately carry the array form.
Possibly related: qwen35moe.feed_forward_length is 0 (the dense FFN is unused in this MoE-only architecture); if the reader divides by or validates against this field it may be a second trap.
Environment
Strata engine: 0.1.64-windows-x86_64-cuda-cuda13\strata.exe
Engine command line:
代码块
strata.exe --serve --native C:\...\qwen3.6-35b-a3b-abliterated.gguf --max-context 131072 --prefill auto --mtp 2
OS: Windows 11 x64, RTX 4060 Laptop 8 GB, driver 580+
File: GGUF v3, Q4_K_M, 22.29 GB, produced by Ollama (llama.cpp conversion pipeline)
Steps to reproduce
Export any Ollama-stored GGUF of a qwen35moe-architecture model (the blob file renamed to .gguf).
Point Strata's "use local GGUF" at it; the UI identifies it as qwen36 UD-Q4_K_XL layout.
Prepare & load → engine exits with qwen35moe.attention.head_count_kv is not a number before "reading the model's weights" completes.
Expected behavior
The qwen36 reader should accept head_count_kv (and friends) in the array form, using llama.cpp's per-layer semantics, instead of aborting with a type error. Tolerating feed_forward_length == 0 for MoE-only checkpoints would also be welcome.
Workaround
None found; the Ollama-side file cannot easily be re-encoded. Using the official in-app Qwen3.6-35B-A3B build works.
[strata-qwen36-ud-q4_k_xl.log](https://github.com/user-attachments/files/35219256/strata-qwen36-ud-q4_k_xl.log)Mehr auf der Site
Links zu Install, Modellen, Releases.