Issues / #2855
#2855 setup: vision encoder not rebuilt for a new GPU arch (sm_61) when the engine is, then fails with an empty error
open · @renaudrenaud · 0 commentaires · Sur GitHub
BenchmarksSetup & installNVIDIA / CUDAModels & quantsLinux
Description
**What happened.** An install built for a Tesla V100 (engine and `strata-vision` for sm_70, Docker image based on `Dockerfile.v100`) was started on two Tesla P40s (sm_61) with `GPUS=0,1`. Setup noticed the new architecture and recompiled the engine: ``` Compiling the engine for sm_61, sm_70 (a card it had no code for; 10-20 minutes, once) ... [ok] engine compiled: /opt/strata/engine-cuda12/strata ``` but it did not rebuild the image encoder. The server then stopped at start: ``` loading the vision encoder ... RuntimeError: the vision encoder did not start: ``` The message after the colon is empty: the encoder exits without printing anything (compute mode was `Default`, so it is not the #1445 case). **Cause.** `cuobjdump --list-elf` on the two binaries: ``` engine-cuda12/strata 62 sm_61, 62 sm_70 engine-cuda12/strata-vision 139 sm_70 ``` `BUILD.json` then listed `archs [61, 70]` and `vision: gpu`, while `strata-vision` still had sm_70 code only. **Workaround.** Rebuilding the encoder with the same call as `Dockerfile.v100` (`setup.cmake_build(ROOT/"tools"/"vision", ..., "strata-vision", [..., "-DSTRATA_VISION_CUDA=ON", "-DCMAKE_CUDA_ARCHITECTURES=61;70", ...])`) and copying it to `engine-cuda12/` fixes it; the server starts with vision on both P40s. **Suggestions.** When setup recompiles the engine for a new architecture, rebuild `strata-vision` for the same list (or check its architectures, e.g. via `vision_src`/archs in `BUILD.json`); and when the encoder exits without a line, name the likely cause (no kernel image for this GPU) instead of an empty message. Environment: Strata 0.1.44.1, CUDA 12.8 toolkit, driver 580.178.04, Linux 7.0, Docker. Benchmark of this setup: #2854. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Sur le site
Liens install, modèles, releases.