Pull requests / #3154
#3154 gentoo-setup: setup.py builds the CUDA and HIP engines on Gentoo
open · @arabek · 0 comentarios · En GitHub
Setup & installAMD / HIPNVIDIA / CUDAModels & quantsWindowsLinux
Descripción
(portage ebuilds, gcc caps, ROCm in /usr)
What changes: setup.py learns the Gentoo paths for both engine builds; the Ubuntu/Debian, Windows and other-Linux flows keep their commands. What it leaves alone: the engine sources, the ready-made engines, and every default on non-Gentoo PCs except the cmake_build build line.
Detection: is_gentoo() is /etc/os-release ID=gentoo with emerge on PATH.
CUDA: dev-util/nvidia-cuda-toolkit's ebuilds cap nvcc's host gcc (GCC_MAX_VER: 12.9 -> 14, 13.4 -> 16); a newer system gcc breaks nvcc there. gentoo_gcc_env picks the newest installed gcc at or below the cap from gcc-config -l and applies that choice's gcc-config -E environment to this process only (PATH first, so gcc, g++ and nvcc's default --ccbin use it); the system's gcc-config pick is never changed and no sudo is used. The toolkit itself installs with sudo emerge dev-util/nvidia-cuda-toolkit.
HIP: portage's ROCm ebuilds install to /usr, so rocm_root's default sysroot is /usr on Gentoo (ROCM_PATH still wins) and hipBLAS is looked for in lib and lib64 (Gentoo keeps it in /usr/lib64); the amdgcn bitcode folder is also sought at root/lib/amdgcn/bitcode. When hipcc or hipBLAS are not installed yet, main() emerges GENTOO_ROCM_EBUILDS (hip, hipcc, rocm-core, rocm-device-libs, rocBLAS, hipBLAS, hipBLASLt; emerge pulls their dependencies).
HIP compiler for CMake: CMake refuses a compiler named hipcc (CMakeDetermineHIPCompiler stops on that name); the wheels' clang is AMD's patched build that takes gfx archs in CUDA mode, while upstream clang needs -x hip for a .cu TU and the same flag on a link line makes it read the .o files as source. hipcc's driver gets both right, so it is wrapped under another name: build-hip/hip-clang++, a one-line script that turns a link call that still carries .cu/.hip sources into a compile to /dev/null.
engine_archs / engine_runs_on: a HIP engine's BUILD.json archs are gfx names, not CUDA arches: engine_archs returns None for backend hip, and an installed engine with no CUDA archs now returns False (compile) instead of True, so a HIP engine no longer passes for a CUDA card's engine, or the other way round.
place_binary: Linux refuses to open a running binary for writing ("Text file busy", errno 26), which setup hit while the server was running the old engine. The new file is copied to <name>.new beside the target and renamed over it with os.replace: the rename unlinks the old file, which is allowed while it runs, and the running server keeps its old copy until it restarts. Used for the HIP engine, the CUDA engine and the image encoder, and for the ROCm runtime binaries placed beside the AMD engine.
fresh_cache_if: CMake re-runs a configure WITHOUT the command-line -Ds when it decides the cache must go (a compiler that changed between configures - Gentoo's hipcc for a folder first configured with clang++), which drops Strata's -DSTRATA_ENABLE_* flags and leaves the target unregistered. A cache whose stored values disagree with the requested -D definitions is removed before configuring, so the next configure is a real first one.
cmake_build: the build step calls ninja directly with -j at the full core count instead of cmake --build at half.
Checked: python -m py_compile setup.py; all tools/test_setup_*.py pass except two failures in test_setup_engine_hash.py that are unchanged from the two previous commits (pre-existing, untouched by this change). The Gentoo paths themselves were ~~not~~ run on Gentoo hardware.
Attributed-to: qwen3.8-flash-next-coder-iq1_m (using Strata) + opencodeEn el sitio
Enlaces a install, modelos, releases.