Issues / #3503
#3503 [Docker] 0.2.1-cuda12 image has no sm_60 engine (P100 restart loop); source Dockerfile's terminal stage lacks git
open · @maxlawrie · 1 comments · View on GitHub
Description
### Area Docker ### Desktop app, Terminal or source? Docker image (and the source Dockerfile as a workaround) ### What happened `docs/DOCKER.md` lists GTX 10 (Pascal) / Tesla P100 / V100 for `ghcr.io/niko1221/strata:0.2.1-cuda12`, but the engine in that image (`0.1.46-linux-x86_64-cuda-cuda12`) has code for sm_75, sm_86, sm_89 and compute_89 only. On 3x Tesla P100 the container stops at start and restarts in a loop: ``` No engine for this PC was found in /opt/strata/components: 0.1.46 linux-x86_64-cuda has code for sm_75, sm_86, sm_89, compute_89, this PC's GPU is sm_60, sm_60, sm_60. [X] The engine in this image does not match this computer's graphics card, or the card is not visible to the container. ``` `strata-feed.json` for v0.2.1 agrees: only the **Windows** cuda12 package (`strata-engine-0.1.46-windows-x86_64-cuda-cuda12.tar.gz`) lists sm_60 / sm_61 / sm_70; the Linux cuda12 package does not. Building from source as a workaround hits a second problem: the root `Dockerfile`'s `terminal` stage installs only `python3 ca-certificates`, but `packaging/package_terminal.py` runs `git rev-parse HEAD` (line 348), so the build fails with `FileNotFoundError: [Errno 2] No such file or directory: 'git'`. (`.git` is in `.dockerignore`, so with `git` installed the call just returns nothing and the COMMIT file is skipped.) **Workaround that works:** build the root Dockerfile with `--build-arg CUDA_IMAGE=nvidia/cuda:12.9.1-devel-ubuntu24.04 --build-arg CUDA_ARCHITECTURES=60 --build-arg BUILD_VISION=0`, and patch it to - add `-DSTRATA_EXPERIMENTAL_SM60=ON` to the engine's cmake line, - change `--min-driver nvidia=580.65 --variant cuda13` to `--min-driver nvidia=575.51 --variant cuda12` (and the build id / `"cuda": "13.0"` in BUILD.json to 12.9), - add `git` to the terminal stage's `apt-get install`. That image runs Flash-Next IQ3_S at 256K on 3x P100 fine (0.1.46 engine, ~+12% decode on code vs our 0.1.42 build, same answers). ### Smallest steps or command that reproduce it 1. On a Linux machine with a Pascal card: `docker run --rm --gpus all -e MODEL=IQ3_S ghcr.io/niko1221/strata:0.2.1-cuda12` 2. The container reports the engine/GPU mismatch above and exits. 3. For the second issue: `docker build --build-arg CUDA_IMAGE=nvidia/cuda:12.9.1-devel-ubuntu24.04 --build-arg CUDA_ARCHITECTURES=60 .` at v0.2.1 fails in `[terminal 4/5]`. ### Reproduction status Reproduces every time ### Strata version 0.2.1 (engine 0.1.46) ### Graphics card, memory and system 3x Tesla P100 16 GB (PCIe), Ryzen 5 2600, 64 GB DDR4, Ubuntu 24.04, NVIDIA driver 580.178.04, Docker with the NVIDIA Container Toolkit ### Model, size and settings Qwen3.8-Flash-Next IQ3_S, 256K context, pictures off, `STRATA_SET="resident-experts kv-resident=32768"`, `GPUS=0,1,2` ### Possible fix Build the Linux cuda12 engine package with `-DSTRATA_EXPERIMENTAL_SM60=ON` and sm_60/61/70 like the Windows one (or narrow the table in `docs/DOCKER.md`), and add `git` to the `terminal` stage of the root `Dockerfile`. A small related note: with `GPUS=all` in the container, the low-RAM resident mode was placed on one card only (GPU 0); naming the cards (`GPUS=0,1,2`) gave the layer split across all three. That may be intended (#364), but `all` reading as "the engine's choice" was a surprise.
Related on this site
Quick links to related pages.