Pull requests / #1588
#1588 serve: answer the request that hit the READY timeout (#1527)
open · @nekomario28 · 0 comments · View on GitHub
Description
Fixes #1527. `STRATA_ENGINE_READY_S` (the #1317 read timeout) kills an engine that never says `READY`, but the request that was loading it ended with no response — the `RuntimeError` raised in `StrataEngine.__init__` matches none of the request path's handlers, so the connection closed unanswered (the reporter's `curl code=000`). On a `--lazy` first start it is worse: `restart()`'s `finally` crashes on `slot_cv`, which exists only after a successful READY, so an `AttributeError` masked the failure as a 400 "the request is malformed". ## Changes - `__init__` raises `EngineDied` instead of bare `RuntimeError` for the start failure. `EngineDied` is a `RuntimeError`, so `restart()`'s retry loop and every existing `except RuntimeError` still catch it — but the request path's `except EngineDied` now answers **503 "the engine did not report READY within N s …; the next request restarts it"**, the same shape as the mid-request silence path that already works. Covers `/v1/chat/completions`, `/v1/messages`, `/v1/responses` and `/v1/load`. - `restart()`'s `finally` skips the `slot_cv` wakeup when a lazy first start never created it — no waiters exist before the first READY anyway. ## Test plan New `serve/test_ready_timeout.py` (runs with `python -m unittest serve.test_ready_timeout`, no GPU/model): - `test_ready_timeout_on_lazy_load_answers_the_request` — silent fake engine, lazy load → **503 with the reason**. Unfixed code: 400 malformed-request. - `test_ready_timeout_on_restart_answers_the_request` — engine answers one request, is killed, then its restart never reports READY → **503**. Unfixed code: the connection closed with no answer (`RemoteDisconnected`), the reported symptom. `serve.test_server` + `serve.test_fatal_recovery` + `serve.test_restart_waiters` + `serve.test_lifecycle`: 299 tests pass (3 unrelated errors — `regex` module not installed here, pre-existing). Not tested: the real engine binary; the fake engines speak the READY/GEN/DONE protocol the tests already rely on.
Related on strata.com
Editorial links to help you install, pick models, or read release notes — not part of the upstream thread.