反馈 / #2251
#2251 STRATA_GGML_DIR with llama.cpp b11580 fails CUDA MMQ precision/template interfaces
open · @Tsury · 2 评论 · 去 GitHub 看
Setup & installAMD / HIPNVIDIA / CUDAModels & quantsLinux
说明
Strata v0.1.43.1 (`a0116b2782a5b82622358c04a06496485d12b2d6`) builds with its shipped llama.cpp dependency (`3cf03257f219afbe7334045ff7c6a06ac68c627d`). This report concerns overriding that dependency through `STRATA_GGML_DIR` with llama.cpp b11580 (`4413a192410fad1f054e9216114087a3e01527a0`). The override's ggml CPU library compiles successfully, but Strata's CUDA MMQ integration does not compile against the newer internal interfaces. Minimal reproduction on Linux with CMake, a C++ compiler, and CUDA 13.3 installed: ```sh git clone https://github.com/Niko1221/Strata.git git -C Strata checkout a0116b2782a5b82622358c04a06496485d12b2d6 git clone https://github.com/ggml-org/llama.cpp.git git -C llama.cpp checkout 4413a192410fad1f054e9216114087a3e01527a0 cmake -S Strata -B build -DCMAKE_BUILD_TYPE=Release \ -DSTRATA_ENABLE_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=120 \ -DSTRATA_GGML_DIR="$PWD/llama.cpp" -DSTRATA_BUILD_TESTS=OFF \ -DSTRATA_MMQ_KQUANTS=OFF -DSTRATA_PREFILL_MMQ=OFF cmake --build build --target strata -j 4 ``` Sanitized decisive diagnostics (ten compiler errors total): ```text src/prefill/moe_mmq.cu(153): error: no instance of overloaded function "ggml_cuda_mmq_get_config" matches the argument list src/prefill/moe_mmq.cu(207): error: no instance of function template "mul_mat_q_case" matches the argument list ``` The second error repeats for the other quantization cases at lines 208–215. The current bridge calls `ggml_cuda_mmq_get_config(type, J, fallback, cc)` and `mul_mat_q_case<type>(ctx, args, stream)`. At b11580 these interfaces require a `ggml_prec` argument and a second template parameter, respectively. The new header explicitly instantiates the ordinary variants with `GGML_PREC_Q8`. Strata already quantizes its activation input through `quantize_mmq_q8_1_cuda`; selecting Q8 preserves that representation rather than enabling the new FP4 activation path. The newer header also exposes `ggml_cuda_mmq_needs_fallback(nrows_x)` for the tile eligibility check. Sources: [Strata bridge](https://github.com/Niko1221/Strata/blob/a0116b2782a5b82622358c04a06496485d12b2d6/src/prefill/moe_mmq.cu#L153), [new MMQ header](https://github.com/ggml-org/llama.cpp/blob/4413a192410fad1f054e9216114087a3e01527a0/ggml/src/ggml-cuda/mmq.cuh#L230). Could Strata support this newer dependency through the existing override, or document the supported override revisions? This is not a failure of the shipped pin or the CPU library itself. An independently built CUDA vision helper using b11580 also builds successfully. We are preparing a narrow maintained bridge update with numerical and task checks while upstream compatibility catches up; no kernel rewrite or llama.cpp source change is proposed.
本站相关内容
相关页面的快捷入口。