Pull requests / #2155
#2155 serve: answer GET /models/sse (llama.cpp's model status feed)
open · @Daniel-Aaron-Bloom · 0 comments · View on GitHub
Server & APIAMD / HIPNVIDIA / CUDADocumentation
Description
Issue: Resolves #2154 ## Summary Answer `GET /models/sse` (llama.cpp's model status feed) so front-ends that watch model status get load and unload events instead of a 404 and endless reconnects. The connection stays open; quiet stretches get the usual SSE-comment keep-alive. ## What changed - `serve/server.py`: the new route (API key as on the other metadata endpoints), a per-watcher queue on `Service` (`sse_lock` / `sse_clients`) and `sse_notify()`, and `status_change` emissions on the transitions the server already has: `loading` and `loaded` around a load, `unloaded` (exit code 0) on unload, `unloaded` with the exit code when the engine had died and is started again. `/models` now builds its entry through a new `Service.model_entry()`, which the SSE `info` reuses; the `/models` answer itself is unchanged. - `serve/test_lifecycle.py`: three tests - the feed end to end over real HTTP (subscribe, load -> `loading`/`loaded` with `info`, unload -> `unloaded`), the quiet heartbeat, the API-key gate. - `docs/DETAILS.md`: the endpoint in the API table and a short paragraph. ## Extra Notes - Purely additive: no existing endpoint changes its answer (the `/models` refactor produces the same JSON) and no default changes, so the default path stays byte-identical. - No C++ files are touched, so there is nothing to build for CUDA, HIP or SYCL. - How I checked: the three new tests run the endpoint over real HTTP; `serve/test_lifecycle.py` 12 pass (9 existing + 3 new), the full `serve/` suite 717 pass / 11 skipped (the skips are pre-existing). Not yet watched live against a client session.
Related on strata.com
Editorial links to help you install, pick models, or read release notes — not part of the upstream thread.