Pull requests / #130

#130 setup: Volta (V100, sm_70) and Turing as an experimental build path

closed · @DingoOz · 0 コメント · GitHub で見る

BenchmarksSetup & installNVIDIA / CUDADocumentationWindowsLinux

本文

Lets setup install Strata on a compute capability 7.x card (Tesla V100, RTX 20) instead of refusing it. The engine's kernels already have pre-sm_80 paths behind `STRATA_EXPERIMENTAL_SM75`; this makes setup use them.

## Changes (`setup.py`)
- **GPU check:** 7.x cards are accepted (below 7.0 is still refused), with a warning that this is experimental and the engine is compiled locally. The ready-made engine has no sm_70/75 code, so the existing "compiling instead" path takes over.
- **Build flag:** `-DSTRATA_EXPERIMENTAL_SM75=ON` is passed when any chosen card is below sm_80.
- **CUDA toolkit:** CUDA 13 dropped Volta. With an sm_70 card, `find_nvcc` picks the newest 12.x toolkit even if a 13.x is also installed, and the automatic install gets 12.9 instead of 13.0.
- **Host compiler (Linux):** CUDA 12.x refuses a distribution's current GCC (for example CUDA 12.4 with GCC 15). When the default `g++` is newer than the toolkit accepts, setup uses the newest `gcc-N`/`g++-N` it does accept for **C, C++ and nvcc together**. Setting only `CMAKE_CUDA_HOST_COMPILER` isn't enough: nvcc adds its GCC's library folder to the link line, so C++ objects built by GCC 15 then fail to link against GCC 12's libstdc++ (`undefined reference to ...basic_string::_M_replace_cold`). An existing `CUDAHOSTCXX` is left alone.

`README.md` gets a short note in "You need".

## Tested
Tesla V100-PCIE-16GB, driver 580.178, CUDA 12.4, Ubuntu with GCC 15 as the default and GCC 12 installed:
- engine 0.1.22 (this branch's `main`) builds for sm_70 through `build_engine`;
- it serves Qwen3.8-Flash-Next IQ2_XS: correct answers, MTP drafts accepted, about 17-24 tok/s decode on short prompts;
- a P100 (sm_60) in the same machine is still reported as not supported.

Not tested: Turing, Windows, the automatic CUDA 12.9 install.

関連リンク

インストール・モデル・リリースへの站内リンク。