Issues / #2251

#2251 STRATA_GGML_DIR with llama.cpp b11580 fails CUDA MMQ precision/template interfaces

open · @Tsury · 2 comentários · No GitHub

Setup & installAMD / HIPNVIDIA / CUDAModels & quantsLinux

Descrição

Strata v0.1.43.1 (`a0116b2782a5b82622358c04a06496485d12b2d6`) builds with its shipped llama.cpp dependency (`3cf03257f219afbe7334045ff7c6a06ac68c627d`). This report concerns overriding that dependency through `STRATA_GGML_DIR` with llama.cpp b11580 (`4413a192410fad1f054e9216114087a3e01527a0`). The override's ggml CPU library compiles successfully, but Strata's CUDA MMQ integration does not compile against the newer internal interfaces.

Minimal reproduction on Linux with CMake, a C++ compiler, and CUDA 13.3 installed:

```sh
git clone https://github.com/Niko1221/Strata.git
git -C Strata checkout a0116b2782a5b82622358c04a06496485d12b2d6
git clone https://github.com/ggml-org/llama.cpp.git
git -C llama.cpp checkout 4413a192410fad1f054e9216114087a3e01527a0
cmake -S Strata -B build -DCMAKE_BUILD_TYPE=Release \
  -DSTRATA_ENABLE_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=120 \
  -DSTRATA_GGML_DIR="$PWD/llama.cpp" -DSTRATA_BUILD_TESTS=OFF \
  -DSTRATA_MMQ_KQUANTS=OFF -DSTRATA_PREFILL_MMQ=OFF
cmake --build build --target strata -j 4
```

Sanitized decisive diagnostics (ten compiler errors total):

```text
src/prefill/moe_mmq.cu(153): error: no instance of overloaded function "ggml_cuda_mmq_get_config" matches the argument list
src/prefill/moe_mmq.cu(207): error: no instance of function template "mul_mat_q_case" matches the argument list
```

The second error repeats for the other quantization cases at lines 208–215. The current bridge calls `ggml_cuda_mmq_get_config(type, J, fallback, cc)` and `mul_mat_q_case<type>(ctx, args, stream)`. At b11580 these interfaces require a `ggml_prec` argument and a second template parameter, respectively. The new header explicitly instantiates the ordinary variants with `GGML_PREC_Q8`. Strata already quantizes its activation input through `quantize_mmq_q8_1_cuda`; selecting Q8 preserves that representation rather than enabling the new FP4 activation path. The newer header also exposes `ggml_cuda_mmq_needs_fallback(nrows_x)` for the tile eligibility check.

Sources: [Strata bridge](https://github.com/Niko1221/Strata/blob/a0116b2782a5b82622358c04a06496485d12b2d6/src/prefill/moe_mmq.cu#L153), [new MMQ header](https://github.com/ggml-org/llama.cpp/blob/4413a192410fad1f054e9216114087a3e01527a0/ggml/src/ggml-cuda/mmq.cuh#L230).

Could Strata support this newer dependency through the existing override, or document the supported override revisions? This is not a failure of the shipped pin or the CPU library itself. An independently built CUDA vision helper using b11580 also builds successfully. We are preparing a narrow maintained bridge update with numerical and task checks while upstream compatibility catches up; no kernel rewrite or llama.cpp source change is proposed.

No site

Links install, modelos, releases.