#4376 [Engine] dsv41: the WMMA expert kernels compile on pre-Volta cards
closed · @lyesrock · 0 comments · View on GitHub
Setup & installServer & APIMulti-GPUAMD / HIPNVIDIA / CUDAModels & quantsLinux
Description
## Problem and result The DeepSeek-V4.1 engine (`strata-dsv41`) does not compile on pre-Volta cards: `mx_set_tc_kernel` (`src/dsv41/ds4_gguf.cu`) and `set_tc_kernel` (`src/dsv41/ds4_iq.cu`) name `nvcuda::wmma` with no arch guard, and `crt/mma.h` declares those names only for `__CUDA_ARCH__ >= 700`. On sm_61 (e.g. 2x Tesla P40) the build fails on the `nvcuda::wmma` declarations before producing any binary. With this change the same source builds on sm_61 and the engine starts and answers there. ## Related discussion none ## Approach The wmma body of the two kernels sits under `#if !defined(__CUDA_ARCH__) || __CUDA_ARCH__ >= 700` inside the kernel, so device passes below sm_70 compile an empty one; the `namespace wm` alias and the `kLdT` constant move inside the guard. This is the pattern `src/prefill/gemm.cu` and `src/prefill/gdn_chunk.cu` already use. No values change: `tc_ok()` (`ds4_gguf.cu`, `ds4_iq.cu`) already keeps cards below sm_70 on the F32 kernels (`mx_set_kernel`, `set_kernel`), so the tensor-core kernels are never launched there. It is a compilation fix, not a perf change. ## Validation All runs ours, one box: 2x Tesla P40 (sm_61), Xeon E5-2620 v4, 92 GiB RAM, Linux; revision 9e0a7319 (this branch, rebased on 5e8a4cca). **Passed:** - sm_61 build, CUDA 12.9 + gcc-14: `cmake -G Ninja -S . -B build-dsv41-sm61 -DCMAKE_BUILD_TYPE=Release -DSTRATA_ENABLE_CUDA=ON -DSTRATA_BUILD_TESTS=OFF -DSTRATA_BUILD_DSV41=ON -DSTRATA_EXPERIMENTAL_SM60=ON -DSTRATA_NATIVE_EXPERTS=ON -DSTRATA_PORTABLE=ON -DCMAKE_CUDA_ARCHITECTURES=61 -DCMAKE_CUDA_COMPILER=/usr/local/cuda-12.9/bin/nvcc -DCUDAToolkit_ROOT=/usr/local/cuda-12.9 -DSTRATA_GGML_DIR=third_party/llama.cpp '-DCMAKE_INSTALL_RPATH=$ORIGIN'` then `ninja -C build-dsv41-sm61 strata-dsv41`: links with 0 errors; `build-dsv41-sm61/strata-dsv41 --version` prints `strata-dsv41 0.1.52 build 9e0a73199`. - sm_89 build (the normal path, guard compiles the full wmma body), CUDA 13.0: same cmake with `-DCMAKE_CUDA_ARCHITECTURES=89-real -DCMAKE_CUDA_COMPILER=/usr/local/cuda-13.0/bin/nvcc -DCUDAToolkit_ROOT=/usr/local/cuda-13.0` (no `STRATA_EXPERIMENTAL_SM60`) then `ninja ... strata-dsv41`: links with 0 errors. - Smoke test on the P40s with the sm_61 binary, pack `deepseek-v4.1-flash-ist-das-2.25bpw`, context 16384, `--tp 2`: one OpenAI chat request (`temperature 0`, `max_tokens 64`) answered "The capital of France is Paris." Coherent output: compilable and functional, not a speed measurement. **Failed:** none observed. (The same sm_61 build without the guard is the failure described under Problem and result: `nvcuda::wmma` is not declared by `crt/mma.h` below sm_70.) **Not run:** HIP and SYCL builds (the touched code sits inside `#if !defined(STRATA_USE_HIP)`, CUDA only); speed measurements (the guarded kernels are never launched on these cards, so nothing measurable changes); a real sm_70 card (the guard covers it, none at hand). ## Limits On a device pass below sm_70 the two kernels compile to empty bodies; launching one there would be a silent no-op, but `tc_ok()` keeps those cards on the F32 paths, so that launch cannot happen. Running the pack on 2x P40 stays bound by the rules and memory bandwidth of a Pascal box; this PR only makes the build possible.
Related on this site
Quick links to related pages.