Pull requests / #2179

#2179 core: read the model's geometry from its GGUF metadata and refuse a shape the kernels are not compiled for

open · @gt50 · 0 comentários · No GitHub

BenchmarksAMD / HIPNVIDIA / CUDAModels & quantsDocumentationWindows

Descrição

## Title
Read the model's geometry from its GGUF metadata and refuse a shape the kernels are not compiled for

## Summary
The engine took its dimensions from `ModelGeometry`'s defaults and read only the expert count, the experts per token and the rope keys from the model file; `check_architecture` compared six keys with numbers written into the guard. Any other key was never looked at: a file that declares, say, an indexer budget of 1024 ran with the kernels' 2048 and said nothing. Now the whole geometry comes from the file's `qwen4exp.*` keys, and a file of a shape the kernels are not compiled for is refused at load, naming the key. For every file the guard accepts nothing changes: the same numbers reach the same kernels, and the output is the same (measured below).

This does not make another architecture run (the forward pass and kernels are still qwen4exp's); it is the step a second model would start from: one place that says what the file declares, one that says what the kernels are compiled for, and the check between them.

## What changed
- **`include/strata/core/geometry.hpp` (new):** `ModelGeometry`, moved out of `layout.hpp` and dependency-free so kernels can include it, with the GGUF key of each field and new fields `n_vocab`, `rms_eps`, `n_rot`, `idx_block`, `idx_top_k`, `n_expert_used`. `kCompiledGeometry` is the shape the kernels are compiled for.
- **`include/strata/artifact/gguf_reader.hpp`:**
  - `read_geometry(f, g)` fills the geometry from the metadata shard (llama.cpp's key names). Another architecture is refused by name first. The six keys the guard always required stay required; any other absent key keeps the compiled value. Keys that state one thing twice must agree: `full_attention_interval` and `compress_ratios` (whose non-zero entry is the pooling block), key and value head widths, routed and shared expert widths, `ssm.inner_size` and `state_size x time_step_rank`. Experts per token 1..64 and at most the expert count.
  - `geometry_unsupported(g)` names the first key whose value the kernels are not compiled for, e.g. `qwen4exp.attention.indexer.top_k = 1024, the kernels are compiled for 2048`. Only the expert count (the GSQ-RCO Coder's 256), the experts per token and the vocabulary may differ.
  - `check_architecture` is the two together; `Qwen4ExpGuard` is gone.
- **Kernels:** the files that fix a dimension at compile time `static_assert` it against `kCompiledGeometry` (fused_gr, qsa_decode_attn, qsa_prompt_attn, qsa_select, native_qsa_indexer, native_qsa_score, fused_gdn, native_gdn, verify_kernels, s2_expert_grouped, native_ple_postops, cpu/expert.hpp; in `sycl/` the same plus qsa_prompt_attn_xmx), so a kernel and the struct cannot drift apart without the build failing.
- **`src/program/generate.cpp`:** the geometry and K come from `read_geometry` and are checked before any weight moves; the native embedding takes `n_embd` / `n_vocab` from it instead of `g0` and a literal 248320.
- **`qsa_shapes(g)` (`layout.hpp`):** one function builds the QSA kernel shapes from the geometry, used by layer, mtp, session, verify, prefill, generate's prompt-chunk sizing and the state fingerprint (each built them by hand before; n_rot, idx_block and idx_top_k came from constants).
- **`sycl/`:** the same changes in its copies of generate, layer, mtp, session, verify and prefill.
- **Tests:** `tests/core/gguf_geometry_test.cpp` (synthetic GGUFs: the full key set, the six required keys, a pruned expert count, every disagreement case, unsupported shapes, a dense `qwen35` file, another architecture; `--real SHARD1` prints a real file's geometry and the verdict). `gguf_fixture.hpp` writes f32 and array keys.

## Extra Notes
- **Measured** on a Radeon 8060S (Ryzen AI Max+ 395, Windows 11), Qwen3.8-Flash-Next UD-IQ4_XS with `--spec 4 --mtp` and the server's other flags, greedy, 256 tokens; this branch and `main` a0116b27 both built here with `tools/hip/build_windows.bat` (gfx1151, engine 0.1.43.1):
  - Prompts of 39 (prose), 48 (code) and 5,568 tokens (a document): the same tokens from both builds, and the same as 0.1.40.2 (before the rebase, the branch matched 0.1.40.2 too).
  - Decode 33 / 47-49 / 39-40 tok/s on both. Reading the 5,568-token prompt: this branch 298, 491, 550 tok/s, `main` 421, 428, 554 tok/s; that spread is run to run on both (on the 0.1.40.2 base, two local builds read it at 404 and 411 tok/s, mean of 3).
  - The same file with `attention.indexer.top_k` patched to 1024 (only the metadata shard changed): `main` loads and answers with 2048; this branch refuses in 0.5 s with the message above.
  - Qwen3.8-27B (UD-Q6_K_XL, Q8_0, a Q4_K_P fine-tune) through `--native`: `architecture is 'qwen35', this engine requires 'qwen4exp'` in 0.5 s, as `main` says.
- **Tests:** ctest 104 of 108 on that build. The four failures need what this PC does not have: `hip_handoff` (the Windows HIP failure in docs/AMD_HIP.md), `ple_parity`, `expert_parity` and `pool_test` (the Q2_0 model and pack fixtures). The GGUF tests also pass built with MSVC `/W4 /WX`.
- **Not tested:** the SYCL tree (no oneAPI compiler here); its changes are line for line the CUDA/HIP tree's. CUDA was not built either; the kernel asserts are host-side `static_assert`s.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

No site

Links install, modelos, releases.