Pull requests / #2509

#2509 [Needs Testing on Multi-GPU, Nvidia, AMD, Intel, MacOS] Web Engine tab: GPU switches for the user's vendor + in-page restart/save

open · @bsorensen110 · 0 Kommentare · Auf GitHub

Setup & installServer & APIMulti-GPUAMD / HIPNVIDIA / CUDASecurityDocumentationWindowsLinux

Beschreibung

**Needs Testing on Multi-GPU and AMD, Intel, MacOS** — the detection, catalog, and restart paths are designed to work on Linux, Windows, and macOS, and are unit-tested for all of them, but the only real hardware exercised here is a single AMD card (Radeon AI PRO R9700, gfx1201, Linux). NVIDIA, Intel, Apple Metal, and multi-GPU layer splits are proven by mocks/tests only.

## What this adds

A web **Engine** tab that shows only the speed/feature switches relevant to the GPU(s) the user actually runs, lets them edit and save those switches into the run config, and **restarts the engine from the page** so the new env/args take effect. Plus a human/AI-readable reference of every switch grouped by GPU manufacturer.

- `docs/GPU_OPTIONS.md` — every env var, engine CLI flag, and setup/CLI option grouped by manufacturer (NVIDIA, AMD, Intel, Apple Metal, cross-vendor, CPU-only), each with what it changes, its gate, and recommended use.
- `docs/ALL_OPTIONS.md` — the master inventory the catalog is generated from (engine flags, server CLI, installer CLI tables).
- `serve/engine_options.py` — the Engine tab's catalog: 351 env switches + 30 engine CLI args (tunable + experimental tiers; debug/internal stay reference-only), plus vendor/arch detection.
- `serve/runconfig.py` — new catalog-allowlisted config keys: `env.NAME`, `arg.--flag` (int/float, bounds-checked), `argflag.--flag`, `argstr.--flag`, `argenum.--flag`.
- `serve/server.py` — `GET /engine-options`, `POST /restart`, and `Service._spawn_from(cfg)` which rebuilds the engine spawn (exe, args, env, cwd, log) from the saved config, so a restart picks up what the page saved.
- `serve/web/` — the Engine tab: scope header (`amd · gfx1201 · strata-<model>.json`), search, experimental toggle, grouped rows with description/gate, save, and a restart banner that appears only when a restart would change the spawn.

## How GPU detection works (cross-platform by construction)

No OS-specific hardware APIs — deliberately. The tab needs the GPU the **engine uses**, not the GPUs the OS enumerates:

1. The run config's `backend` key is the primary signal: `cuda`→nvidia, `hip`→amd, `sycl`→intel, `metal`→apple.
2. The engine's own startup lines in its log (which the server writes on every OS) refine it: `gfx\d{3,4}`, `sm_\d{2,3}`, `compute capability X.Y` (two-digit majors included, e.g. `12.0`→`sm_120`), `Arc …`, `Apple M… Pro/Max/Ultra`.
3. **Multi-GPU layer splits**: every architecture line in the log tail is collected, so a mixed NVIDIA+AMD split lists both vendors' switches (union, deduped). A mixed split shows `nvidia + amd` in the scope header.

This works identically on Linux, Windows, and macOS because it reads the config and the engine's log — no WMI/`system_profiler`/`/sys` branches, no driver-dependent libraries, no extra permissions.

## Safety

- The catalog is the allowlist: only listed keys are accepted by `runconfig.check()`; unlisted names, control characters, oversized values, out-of-bounds numbers, and bad enum choices are refused.
- `/restart`, `/unload`, `/load`, `/config` accept JSON only from Strata's own page (Origin/CSRF check) — a cross-site form POST gets 403.
- Restart is busy-safe: while a request or a load holds the turn, `/restart` returns 409 (`{"status": "busy"}`) at once and changes nothing; it does not wait for the turn. `restart_supported()` is false for mock/replica setups, which get an honest 400.
- A `None` value means "engine default": the key is removed from the config (including setup-written flags like `--kv`).
- Config writes are atomic (temp file + backup + `os.replace`).

## Tests

Full serve suite on top of current `main`: **451/451 OK** (`serve.test_runconfig serve.test_server serve.test_lifecycle serve.test_security serve.test_frontend serve.test_monitor serve.test_slots serve.test_replicas`).

New coverage: catalog sanity; vendor mapping for all four backends; arch regexes (gfx, sm_ incl. `sm_120`, compute capability, Arc, Apple M); multi-GPU union (`detect_archs`, `detect_vendors`, mixed `nvidia + amd` scope over HTTP, no duplicate keys); Apple Metal options + config save round-trip; env/arg round-trips incl. float args (`--spec-min-p 0.65`); refusals (bounds, enums, types); restart guarded (evil Origin 403, mock 400, busy 409 with config untouched); Metal-config restart rebuilds the spawn with the saved env.

## Verified / not verified

- Verified on real hardware: Linux + AMD gfx1201, single GPU — tab scope, vendor filtering, save, live service health.
- Verified by mocks (Linux): Apple Metal config (`backend: metal` + `Apple M4 Pro` startup line → `apple` scope, Metal switches shown, AMD/NVIDIA hidden, save round-trip), mixed NVIDIA+AMD split, mock restart paths.
- **Not verified on real hardware: NVIDIA, Intel, macOS, and any multi-GPU machine.** The in-page restart button is fake-engine-tested, not clicked against a production GPU. Windows process-group handling on restart (`creationflags`) is not exercised.

## Reproduce

```
python -m serve.server --config strata-<model>.json   # open the web page, Engine tab
python -m unittest serve.test_runconfig serve.test_server serve.test_lifecycle \
    serve.test_security serve.test_frontend serve.test_monitor serve.test_slots serve.test_replicas
```

## Known limitations

- Detection reads the last 4 MiB of the engine log; a banner older than that (log rotated away) falls back to the `backend` key alone.
- The Metal setup flags (`metal/setup_mac.py`) are documented in `docs/GPU_OPTIONS.md` but are bootstrap choices, not live engine env/args, so they are not Engine-tab-editable.
- `serve/engine_options.py` is generated from the inventory in `docs/ALL_OPTIONS.md`; regenerate after upgrades (the generator lives in the contributor's tooling, not in this repo).

Mehr auf der Site

Links zu Install, Modellen, Releases.