Pull requests / #2308
#2308 docker: complete deployment bundle with a management console
open · @jazzlee008 · 0 comentarios · En GitHub
BenchmarksServer & APIAMD / HIPNVIDIA / CUDAModels & quantsSecurityDocumentation
Descripción
## Summary Adds `docker/` — a self-contained deployment bundle for Strata: the engine image, an nginx TLS + authorization entry point, and a FastAPI manager serving a monitoring console and recording inference metrics. The upstream repo already ships a root-level `Dockerfile` and `docker-entrypoint.sh` for a single inference container. This adds the pieces around it that a shared or long-running deployment needs: a gateway that terminates TLS and checks credentials, a manager that exposes metrics and lets you manage keys, and a Compose file that wires the three together. ## What's included ``` docker/ ├── docker-compose.yml three services: nginx / manager / strata ├── Dockerfile engine image (compiled from the Strata sources) ├── Dockerfile.nginx gateway image ├── nginx.conf TLS + auth_request + SSE/WebSocket forwarding ├── manager/ FastAPI console (app.py, Dockerfile, static/) ├── scripts/ cert generation, key bootstrap, build-context prep └── tests/ i18n coverage check ``` Only `nginx` publishes a host port. `manager` and `strata` are reachable on the internal network only, so the engine never depends on its own authorization. ## The stack | Service | Role | | --- | --- | | `nginx` | The only external entry point. TLS termination, `auth_request` authorization, SSE and WebSocket forwarding | | `manager` | System metrics (SQLite), inference metrics, API key store, user sessions, container lifecycle, web shell, console UI | | `strata` | The CUDA inference engine. GPU-exclusive, internal network only | ## Design notes worth reviewing **The build context is assembled by a script.** The engine Dockerfile contains `COPY . .`, so a build needs the full source tree. Pointing the build context at the repository root would also send sibling secrets (TLS private keys, API key files) to the Docker daemon, even though they are never copied into the image. `scripts/prepare-build.sh` copies the sources plus this directory's Dockerfile into a clean `.build-context/`, so the context never contains credentials. **The console is bilingual.** English by default; Chinese when the browser asks for it; a header button overrides the choice. Runtime-generated strings go through `t()`, the rest are translated by walking the DOM on load. `node tests/i18n-coverage.js` builds a DOM from the real `index.html`, runs `applyLang()`, and fails if any Chinese text survives — it currently reports 0. **WebSockets authorize themselves.** `/ws/` deliberately bypasses nginx `auth_request`: the subrequest does not inherit the original query string, so `?api_key=` (the only way a browser can carry a credential on a WebSocket) cannot be verified at that layer. The manager's WebSocket endpoints check the session cookie or the query parameter instead. **Shell access is opt-in.** The terminal tab does not connect on load; it requires an explicit click on *Connect*, shows a status indicator, and disconnecting terminates the shell session. **This Dockerfile tracks the upstream one.** Same build logic, with two local adjustments: China-friendly apt/PyPI mirrors, and removal of the `# syntax` directive (it makes BuildKit fetch a frontend image from docker.io). Both are easy to drop; CONTRIBUTING.md documents how to re-sync from upstream. ## Verification Built and run on an RTX 4090 (24 GB) with `IQ3_S`, 256K context, vision enabled: - `docker compose config` — clean - `docker compose up -d --build` — all three containers reach `healthy` - Unauthenticated `/api/*` and `/v1/*` → 401; UI shell, `/health` open - `node tests/i18n-coverage.js` → 0 untranslated strings - Inference: 125B MoE, ~100 tok/s steady state, 256K context loaded ## Known limitations Happy to take feedback on any of these: 1. **The web shell runs as root.** It requires an explicit connect click, but command auditing, session timeouts and a non-root user are not implemented. 2. **The engine is single-slot**, so concurrent requests queue. This is the engine's design, not something the bundle introduces. 3. **System metrics are sampled every 10 seconds** (`SAMPLE_INTERVAL`). 4. **Self-signed TLS.** Clients must trust the certificate manually. ACME or mounting an externally issued certificate would be a natural next step. 5. **`prepare-build.sh` is a shell script.** It could be a `make` target or a Compose-native mechanism if the project prefers. ## Files - `README.md` — user-facing documentation - `CONTRIBUTING.md` — checklist, conventions, known limitations, upstream sync
En el sitio
Enlaces a install, modelos, releases.