Pull requests / #3595

#3595 [Engine] the experimental CUDA 12 build runs on the Maxwell cards (sm_50)

open · @msprengholz · 0 comments · View on GitHub

BenchmarksSetup & installAMD / HIPNVIDIA / CUDALinux

Description

## The problem

CUDA 13 dropped Maxwell, so a Maxwell card needs the experimental CUDA 12 engine. That engine refused every
card below compute capability 6.0, although the kernels compile for Maxwell and a GTX TITAN X (sm_52) is a
12 GB card with enough memory for a small model. The gate was the only thing in the way.

## What this changes

- `CMakeLists.txt`: the experimental build's lowest architecture is 5.0 (it was 6.0). The release CUDA 13
  engine still needs 7.5.
- `setup.py`: setup's card choice admits the Maxwell cards (sm_50/52) for the CUDA 12 engine, and still
  refuses Kepler (sm_35), for which no engine code exists.
- `src/core/device.cu`: the engine's own check says "5.0 or newer" in the experimental build.
- `tools/test_setup_choices.py`, `tools/test_setup_older_gpus.py`: the tests follow (50 and 19 tests).

Nothing else changes: the release engine, the HIP engine, the SYCL engine and the Metal engine are untouched.

## Validation

Environment: GeForce GTX TITAN X (sm_52, 12,288 MiB), driver 580.178.04, Linux 6.2 (CachyOS), CUDA 12.9 at
/opt/cuda, host compiler g++-14. Engine built with
`-DCMAKE_CUDA_ARCHITECTURES=50 -DSTRATA_EXPERIMENTAL_SM60=ON` (the `build-cuda12` folder).

**Passed**

- `python tools/test_setup_choices.py` - 50 tests, OK.
- `python tools/test_setup_older_gpus.py` - 19 tests, OK.
- `python tools/test_setup_configs.py` - OK.
- `cmake --build build-cuda12 -j 8` - clean, no warnings from the changed files.
- On the card: the engine loads the Gyro-S model, serves it and answers (the profile in the next PR:
  `bench/results/2026-10-21-gyro-titanx`, a real request, 201 tokens decoded, 77 s).

**Failed**

- None.

**Not run**

- The HIP, SYCL and Metal engines: this change does not touch their sources, and this PC has no ROCm and no
  Intel toolchain. The CUDA 13 release engine was not built here either (only CUDA 12.9 is installed).

## The next pull request

The profile of this card is in the next PR: the GPU's kernels under Nsight Systems, the bottleneck analysis
and the one change that measured faster there (the dense projections' FP16 GEMMs as `cublasSgemm` on FP32
copies, opt-in `STRATA_DENSE_F32=1`, 1.18-1.65x for the big shapes, bitwise on 5,201 chunk sizes).

Related on this site

Quick links to related pages.