Issues / #3217
#3217 [Engine] Current GGML override omits precision arguments in the trellis MMQ overlay
open · @Tsury · 0 commentaires · Sur GitHub
Description
Using a current upstream GGML through `STRATA_GGML_DIR`, the new trellis MMQ overlay still emits calls from the older precision API. The CUDA build fails before any model starts. Environment: Strata `e88ee39d1d6be6e3364183af3b39b62a7ed6f211` (engine 0.1.45.3), upstream llama.cpp `0ce8cd11416c8840498e6701bcbe39ad4b858c2b` (b11739), Linux, CUDA 13.3, SM120. This was reproduced in a maintained build with existing adapters for the other current-GGML MMQ callers. It is an override-compatibility report, not a claim that Strata's pinned `3cf03257` default is broken. No weights are required to reach the compile error. The remaining callers are in `cmake/strata_ggml_tq_overlay.cmake` and `third_party/ggml-tq/tq-mmq-tiles.cuh`: - Generated DP4A and MMA write-back callbacks omit `prec_src1`. - The generated `ggml_cuda_mmq_vec_dot_q8_0_16_q8_1_mma` callback omits `prec_src1`. - The trellis loader calls `ggml_cuda_mmq_get_nthreads`, `ggml_cuda_mmq_get_I`, and `ggml_cuda_mmq_get_sram_stride` without the new precision argument. The current upstream IQ2_XS loader uses `GGML_PREC_Q8` for these three calls. Representative errors: ```text mmq.cuh: error: no instance of constructor ggml_cuda_mmq_util_funcs matches the argument list argument types are: (int, <unknown-type>, <unknown-type>, <unknown-type>) tq-mmq-tiles.cuh: error: too few arguments in function call ggml_cuda_mmq_get_nthreads(type, J, fallback) ``` Locally, passing `prec_src1` to the three generated callbacks and `GGML_PREC_Q8` to the three loader helpers, alongside the existing adapters, lets the current-GGML text-engine target build. All failed attempts were retained. Both text and vision builds now complete. Selected native text, image-history recovery, Pi/Hermes image/tool writes, all eight tool-sanity cases, and 122K-input dispersed retrieval pass on this maintained SM120 build. These are brief checks; the selected model does not use trellis weights. HIP/SYCL builds, trellis inference, and compatibility with the pinned GGML remain untested. No performance claim follows. A useful upstream fix could adapt the overlay to the actual GGML signature, or reject an incompatible override clearly during configuration. The current text anchors accept this source but do not catch the later signature mismatch.
Sur le site
Liens install, modèles, releases.