Pull requests / #2155

#2155 serve: answer GET /models/sse (llama.cpp's model status feed)

open · @Daniel-Aaron-Bloom · 0 comentários · No GitHub

Server & APIAMD / HIPNVIDIA / CUDADocumentation

Descrição

Issue: Resolves #2154

## Summary

Answer `GET /models/sse` (llama.cpp's model status feed) so front-ends that watch model status get load and unload events instead of a 404 and endless reconnects. The connection stays open; quiet stretches get the usual SSE-comment keep-alive.

## What changed

- `serve/server.py`: the new route (API key as on the other metadata endpoints), a per-watcher queue on `Service` (`sse_lock` / `sse_clients`) and `sse_notify()`, and `status_change` emissions on the transitions the server already has: `loading` and `loaded` around a load, `unloaded` (exit code 0) on unload, `unloaded` with the exit code when the engine had died and is started again. `/models` now builds its entry through a new `Service.model_entry()`, which the SSE `info` reuses; the `/models` answer itself is unchanged.
- `serve/test_lifecycle.py`: three tests - the feed end to end over real HTTP (subscribe, load -> `loading`/`loaded` with `info`, unload -> `unloaded`), the quiet heartbeat, the API-key gate.
- `docs/DETAILS.md`: the endpoint in the API table and a short paragraph.

## Extra Notes

- Purely additive: no existing endpoint changes its answer (the `/models` refactor produces the same JSON) and no default changes, so the default path stays byte-identical.
- No C++ files are touched, so there is nothing to build for CUDA, HIP or SYCL.
- How I checked: the three new tests run the endpoint over real HTTP; `serve/test_lifecycle.py` 12 pass (9 existing + 3 new), the full `serve/` suite 717 pass / 11 skipped (the skips are pre-existing). Not yet watched live against a client session.

No site

Links install, modelos, releases.