Pull requests / #102

#102 feat: optional RAM and VRAM guard for the server it launches

closed · draft · @midhatn · 0 コメント · GitHub で見る

Server & APIAMD / HIPModels & quantsDocumentationWindows

本文

This adds an optional launcher for machines where Strata fits at startup but another application can later consume the remaining RAM or VRAM. It checks configured floors before launch and while the server loads/runs, and stops only the server it owns when a floor is crossed or required telemetry disappears.

Usage and limitations are in `docs/RESOURCE_GUARD.md`. No existing launcher or default changes. The GPU index is passed to both NVML and the server, so monitoring follows the selected physical card. It uses available physical RAM and NVML's actual free VRAM.

Windows containment uses a Job Object: a small worker joins the job before spawning the frontend, and the engine inherits membership. POSIX uses a new process group. No process-name scans or termination of other applications. The stop is deliberately a forced teardown, with a nonzero exit status, not a context-saving shutdown; an in-flight request fails. Sampling cannot guarantee prevention of OOM or reserve memory.

Validation on Windows against current main (`c1e9033`, v0.1.20):

- Six tests passed (`python -m unittest serve.test_resource_guard -v`).
- Real subprocess/descendant cleanup on simulated pressure and missing telemetry, while an unrelated process stays alive.
- Frontend exit cleans up its remaining child; ordinary exit status is preserved.
- A real mock Strata HTTP server becomes healthy, then is stopped and releases its listener.
- Live RAM/NVML probes work, and an impossible floor refuses launch without allocating memory.

No full-model pressure test or POSIX runtime validation is claimed. Opening as a draft for feedback on the opt-in wrapper approach and the forced-stop policy before expanding platform coverage. This complements startup fitting and the admission checks discussed in #57; it does not implement conversation caching or expert tiering.

関連リンク

インストール・モデル・リリースへの站内リンク。