#4376 [Engine] dsv41: the WMMA expert kernels compile on pre-Volta cards

closed · @lyesrock · 0 comentários · No GitHub

Setup & installServer & APIMulti-GPUAMD / HIPNVIDIA / CUDAModels & quantsLinux

Descrição

## Problem and result

The DeepSeek-V4.1 engine (`strata-dsv41`) does not compile on pre-Volta cards: `mx_set_tc_kernel` (`src/dsv41/ds4_gguf.cu`) and `set_tc_kernel` (`src/dsv41/ds4_iq.cu`) name `nvcuda::wmma` with no arch guard, and `crt/mma.h` declares those names only for `__CUDA_ARCH__ >= 700`. On sm_61 (e.g. 2x Tesla P40) the build fails on the `nvcuda::wmma` declarations before producing any binary. With this change the same source builds on sm_61 and the engine starts and answers there.

## Related discussion

none

## Approach

The wmma body of the two kernels sits under `#if !defined(__CUDA_ARCH__) || __CUDA_ARCH__ >= 700` inside the kernel, so device passes below sm_70 compile an empty one; the `namespace wm` alias and the `kLdT` constant move inside the guard. This is the pattern `src/prefill/gemm.cu` and `src/prefill/gdn_chunk.cu` already use. No values change: `tc_ok()` (`ds4_gguf.cu`, `ds4_iq.cu`) already keeps cards below sm_70 on the F32 kernels (`mx_set_kernel`, `set_kernel`), so the tensor-core kernels are never launched there. It is a compilation fix, not a perf change.

## Validation

All runs ours, one box: 2x Tesla P40 (sm_61), Xeon E5-2620 v4, 92 GiB RAM, Linux; revision 9e0a7319 (this branch, rebased on 5e8a4cca).

**Passed:**

- sm_61 build, CUDA 12.9 + gcc-14: `cmake -G Ninja -S . -B build-dsv41-sm61 -DCMAKE_BUILD_TYPE=Release -DSTRATA_ENABLE_CUDA=ON -DSTRATA_BUILD_TESTS=OFF -DSTRATA_BUILD_DSV41=ON -DSTRATA_EXPERIMENTAL_SM60=ON -DSTRATA_NATIVE_EXPERTS=ON -DSTRATA_PORTABLE=ON -DCMAKE_CUDA_ARCHITECTURES=61 -DCMAKE_CUDA_COMPILER=/usr/local/cuda-12.9/bin/nvcc -DCUDAToolkit_ROOT=/usr/local/cuda-12.9 -DSTRATA_GGML_DIR=third_party/llama.cpp '-DCMAKE_INSTALL_RPATH=$ORIGIN'` then `ninja -C build-dsv41-sm61 strata-dsv41`: links with 0 errors; `build-dsv41-sm61/strata-dsv41 --version` prints `strata-dsv41 0.1.52 build 9e0a73199`.
- sm_89 build (the normal path, guard compiles the full wmma body), CUDA 13.0: same cmake with `-DCMAKE_CUDA_ARCHITECTURES=89-real -DCMAKE_CUDA_COMPILER=/usr/local/cuda-13.0/bin/nvcc -DCUDAToolkit_ROOT=/usr/local/cuda-13.0` (no `STRATA_EXPERIMENTAL_SM60`) then `ninja ... strata-dsv41`: links with 0 errors.
- Smoke test on the P40s with the sm_61 binary, pack `deepseek-v4.1-flash-ist-das-2.25bpw`, context 16384, `--tp 2`: one OpenAI chat request (`temperature 0`, `max_tokens 64`) answered "The capital of France is Paris." Coherent output: compilable and functional, not a speed measurement.

**Failed:** none observed. (The same sm_61 build without the guard is the failure described under Problem and result: `nvcuda::wmma` is not declared by `crt/mma.h` below sm_70.)

**Not run:** HIP and SYCL builds (the touched code sits inside `#if !defined(STRATA_USE_HIP)`, CUDA only); speed measurements (the guarded kernels are never launched on these cards, so nothing measurable changes); a real sm_70 card (the guard covers it, none at hand).

## Limits

On a device pass below sm_70 the two kernels compile to empty bodies; launching one there would be a silent no-op, but `tc_ok()` keeps those cards on the F32 paths, so that launch cannot happen. Running the pack on 2x P40 stays bound by the rules and memory bandwidth of a Pascal box; this PR only makes the build possible.

No site

Links install, modelos, releases.