Pull requests / #130
#130 setup: Volta (V100, sm_70) and Turing as an experimental build path
closed · @DingoOz · 0 comentarios · En GitHub
BenchmarksSetup & installNVIDIA / CUDADocumentationWindowsLinux
Descripción
Lets setup install Strata on a compute capability 7.x card (Tesla V100, RTX 20) instead of refusing it. The engine's kernels already have pre-sm_80 paths behind `STRATA_EXPERIMENTAL_SM75`; this makes setup use them. ## Changes (`setup.py`) - **GPU check:** 7.x cards are accepted (below 7.0 is still refused), with a warning that this is experimental and the engine is compiled locally. The ready-made engine has no sm_70/75 code, so the existing "compiling instead" path takes over. - **Build flag:** `-DSTRATA_EXPERIMENTAL_SM75=ON` is passed when any chosen card is below sm_80. - **CUDA toolkit:** CUDA 13 dropped Volta. With an sm_70 card, `find_nvcc` picks the newest 12.x toolkit even if a 13.x is also installed, and the automatic install gets 12.9 instead of 13.0. - **Host compiler (Linux):** CUDA 12.x refuses a distribution's current GCC (for example CUDA 12.4 with GCC 15). When the default `g++` is newer than the toolkit accepts, setup uses the newest `gcc-N`/`g++-N` it does accept for **C, C++ and nvcc together**. Setting only `CMAKE_CUDA_HOST_COMPILER` isn't enough: nvcc adds its GCC's library folder to the link line, so C++ objects built by GCC 15 then fail to link against GCC 12's libstdc++ (`undefined reference to ...basic_string::_M_replace_cold`). An existing `CUDAHOSTCXX` is left alone. `README.md` gets a short note in "You need". ## Tested Tesla V100-PCIE-16GB, driver 580.178, CUDA 12.4, Ubuntu with GCC 15 as the default and GCC 12 installed: - engine 0.1.22 (this branch's `main`) builds for sm_70 through `build_engine`; - it serves Qwen3.8-Flash-Next IQ2_XS: correct answers, MTP drafts accepted, about 17-24 tok/s decode on short prompts; - a P100 (sm_60) in the same machine is still reported as not supported. Not tested: Turing, Windows, the automatic CUDA 12.9 install.
En el sitio
Enlaces a install, modelos, releases.