Pull requests / #2336

#2336 sycl serve: opt-in non-root engine container and read-only data mount

open · @leadgtr7 · 0 comentários · No GitHub

BenchmarksServer & APIAMD / HIPNVIDIA / CUDAModels & quantsDocumentation

Descrição

**What it changes:** two opt-in settings in `sycl/serve/strata-sycl.sh`, documented in its header:
- `STRATA_SYCL_USER=uid:gid` runs the engine container as that user (`--user`, `HOME=/tmp`) and adds the groups that own `/dev/dri/*` (`--group-add`), so the engine can still open the GPU.
- `STRATA_SYCL_RO=1` mounts the data root at `/work` read-only.

**Why:** the container runs as root with the whole data root (models, packs, the checkout) mounted read-write, though the engine only reads it while serving. With these set, a fault or a bad input in the engine cannot write to the model files or the checkout. docs/A770_SPEED_SPEC.md also notes the root container stops `gputop` from attributing the engine; a user-owned engine fixes that.

**What it leaves alone:** with neither variable set, the `docker run` arguments are identical to main (checked with a `docker` stub that prints them, sorted, `env -i`). Builds still need a writable mount; this is for serving only. CUDA/HIP untouched.

**How it was checked:**
- The `docker` stub: default identical to main; with `STRATA_SYCL_USER=65534:65534 STRATA_SYCL_RO=1` the arguments gain `--user 65534:65534`, `-e HOME=/tmp`, one `--group-add` per `/dev/dri` group and `<root>:/work:ro`.
- On one Arc Pro B65 (xe driver, render group access through `--group-add`), a v0.1.44 engine built for `bmg-g31` ran Swift 1.5 IQ3_S with `STRATA_SYCL_USER=65534:65534 STRATA_SYCL_RO=1` (`docker inspect`: user `65534:65534`, `/work` with `RW=false`) through every arm of our benchmark runs (36.8 tok/s decode), a 61,052-token prompt (finish `stop`) and a `"parallel": 2` run with four concurrent requests. The same two settings have run in our daily server since 0.1.41; when we turned them on there, decode stayed within run-to-run noise on the same build. We did not repeat that root vs non-root comparison on 0.1.44.

**Not tested:** rootless Docker, Podman, two-card setups.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

No site

Links install, modelos, releases.