Issues / #3421
#3421 [Other] Docker 0.2.x: no way to run the engine without the `strata launch` wrapper (re-downloads existing GGUFs, forces an API key)
open · @nullata · 2 commentaires · Sur GitHub
Setup & installServer & APIModels & quantsSecurityDocumentation
Description
### Area
[Other] (setup and installation: the Docker image's entrypoint)
### Desktop app, Terminal or source?
Docker
### What happened
I run Strata as a backend for an orchestrator. The orchestrator manages model files, ports, auth, GPU assignment, restarts and the UI. All it needs from Strata is: **here are the weights, serve them on a port.**
The 0.1.x image did that. After rebuilding from `main` (0.2.x), a container started with the GGUF shards already mounted where 0.1.x read them (`/data/models/<tag>/<original name>`) **ignored them and started downloading the model again**. The download is ~70-110 GB. There was no "files not found" error. It just downloaded.
`docs/DOCKER.md` now says *"The old volume layout (`/data/config`, `/data/models`) is not read"*. That was documented after the fact under "From the older image", with no deprecation period. If you pass no `MODEL`, the image picks a model on its own and downloads that too.
**Expected:** an inference server never fetches tens of gigabytes because a path moved. It fails loudly instead.
The re-download is the part I saw. The rest of this report comes from reading the 0.2.x source to work out why. Those items are **unconfirmed static analysis**, marked as such, with file references. They all have the same cause: the 0.2.x image makes `strata launch` (a desktop/TUI flow with presets, tuning, access modes, key generation, a status loop and its own downloader) the only door into the engine, and none of it can be turned off.
#### 1. Mandatory API key, enforced by the wrapper and not the server (unconfirmed static analysis)
`serve/server.py` handles no-key operation correctly: Host-header allow-listing via `allowed_hosts` / `STRATA_ALLOWED_HOSTS`, origin checks against DNS rebinding, and `/health` outside the auth gate. That's a well-designed keyless mode for a private network.
`strata_cli/cli.py` `_launch_plan()` overrides it:
```python
if host not in ("127.0.0.1", "localhost", "::1") and not has_key:
raise ProfileError("non-loopback serving requires an API key in the run config or STRATA_API_KEY")
```
`docker/entrypoint.sh` hard-codes `--access lan`. So inside a container, where loopback-only is useless, a key is always required. If you don't give one, it makes one up and stores it in the volume (`/data/state/api-key`). Going by the code, any client that doesn't send that key gets `401`.
A container on an internal Docker network behind a proxy that does its own auth doesn't need a second key, and that isn't Strata's decision to make. The server already has the right primitive (`STRATA_ALLOWED_HOSTS`). The launcher overrules the server's own security model.
#### 2. The 0.1.x env vars were dropped without a replacement (from `docs/DOCKER.md` and `entrypoint.sh`)
`HOST`, `LOW_RAM`, `REINSTALL`, `LAYER_SPLIT`, `RESIDENT_BUDGET_GIB`, `KV_STREAMING` and `CONFIG` are now *"ignored with a warning"*.
- `REINSTALL=1` was the only explicit way to rebuild a model's config after changing context, vision, KV or low-RAM. What replaced it is presets: state saved in the volume that decides which settings win. The entrypoint even deletes your run configs when `GPU`/`GPUS` changes, because `launch --gpu` otherwise doesn't apply to a model that's already set up.
- `LOW_RAM` matters because setup reads the **host's** `/proc/meminfo` and not the cgroup limit. That's still true (the entrypoint now only *warns* about a memory limit), and the override is gone from the image's interface. `setup.py --low-ram` itself still exists.
Warning about an env var and then ignoring it is not a migration path. Either honour them for a release or refuse to start.
#### 3. The container polices its own lifecycle (unconfirmed static analysis)
`docker/entrypoint.sh` runs a loop that calls `strata launch --status` every 10 s and exits after three misses. Liveness belongs to the orchestrator (Docker healthchecks, k8s probes, or whatever started the container), not a shell loop inside it. I haven't seen it fire. But nothing in the loop distinguishes "server gone" from "status probe slow", and with one request at a time and long prefills, that difference matters.
#### 4. Presets, tuning, model "recommendation", access modes, key files
`PRESET`, `TUNE`/`RETUNE`, "Strata picks the model it recommends for this card", `--access local|lan|cloudflare|tailscale`, `--save-preset`, `/data/state/api-key`. These are desktop-app features. In a container they're state the operator didn't ask for, can't see without `docker exec`, and that can change behaviour from one start to the next.
#### 5. Internals moved with no stable surface
`setup.py` moved from `/opt/strata` to `/opt/strata/terminal`, and its tables gained families that don't fit their previous shape (`"ready": True` with no `hf`/`file`; single-file `gyro` without `{i}`). That's fair game for internals. But there is no **supported** machine-readable alternative for an integrator (a catalogue command, a structured setup-progress file), so integrators scrape, and break.
#### Is Strata an engine or an appliance?
Strata currently presents itself as three things at once:
1. **An inference engine** (`serve/server.py` and the engine component): fast, careful, with a sane security model.
2. **An installer** (`setup.py`): reasonable as a one-shot tool.
3. **A consumer appliance** (`strata launch`, presets, tuning, access modes, generated keys, self-supervision).
The 0.2.x Docker image makes (3) the only door into (1). That's the wrong way round for a **container image**, whose users are by definition operators who already run something on top of it.
What makes this worse: **everything an operator needs is still in the image.** `setup.py --setup --yes --no-start --gguf-dir ... --low-ram ...` still accepts every option the wrapper drops. `serve/server.py --engine strata --config ... --host 0.0.0.0` runs keyless with `STRATA_ALLOWED_HOSTS` (it warns, it doesn't refuse). And `entrypoint.sh` passes through any command that isn't `run`. The engine was never the problem. The wrapper is the problem, because it's the only supported way in.
#### What would fix it
In order of preference:
1. **A supported server-only mode**, e.g. `docker run strata serve` or `MODE=engine`. It runs setup with `--gguf-dir` if needed, then `serve/server.py` on that run config. It never downloads model files, never generates keys, and respects `STRATA_ALLOWED_HOSTS`. `strata launch` stays the default for people who want the appliance.
2. **Make the launcher honour the server's own options**: keyless with `STRATA_ALLOWED_HOSTS`, a `NO_DOWNLOAD=1` that turns a missing file into a hard error, and a way to turn off the status loop.
3. **A documented, versioned integration surface**: a JSON catalogue command, a structured setup-progress file, and a promise that the env-var contract only changes at a major version, with a deprecation window.
### Smallest steps or command that reproduce it
1. With a 0.1.x image, set up a model whose shards sit in the mounted folder: `-v /path/to/IQ2_XS:/data/models/IQ2_XS -e FAMILY=qwen -e MODEL=IQ2_XS`. No download happens.
2. Rebuild the image from `main` (0.2.x) and start it with the same mounts and env.
3. The container starts downloading the model.
Workaround in use: override the entrypoint, run `/opt/strata/terminal/setup.py` in Control mode with `--gguf-dir`, then start `serve/server.py` directly. This depends on internal paths that may move again in the next release.
### Reproduction status
Reproduced locally: the re-download. Items 1 and 3: unconfirmed static analysis, as marked.
### Strata version
`9ae93f7` (0.2.x, built from `main`; 0.1.45 worked)
### Graphics card, memory and system
irrelevant
### Model, size and settings
irrelevant
### Logs
irrelevant
Sur le site
Liens install, modèles, releases.