Pull requests / #1183

#1183 serve: release batch control after terminal native ERR

closed · @hulkbig · 0 Kommentare · Auf GitHub

Server & APILinux

Beschreibung

Fixes #1059

A native validation rejection emits `ERR` and returns to the input loop without `DONE` or `BADM`. With batch control active, the Python generator's cleanup then sends STOP and waits up to 300 seconds for a terminal line that will never arrive, holding the control lock and delaying the next request.

Raise a dedicated `EngineRequestError` (still a `ValueError`) for an ERR consumed by `_control`, and mark that command terminal before cleanup. Malformed protocol values still drain; cancellation, consumer close, fatal errors and restart cleanup retain their existing behavior. The production drain timeout is unchanged.

Regression tests use real HTTP and pipes with a scripted native process. A rejection is gated until a second request is queued; both requests must finish promptly, release the control lock and reserved slot, and preserve another active slot. They also exercise the existing malformed-protocol, consumer-close and fatal-error cleanup paths.

Validation on macOS arm64 / Python 3.12.12: `python -m unittest serve.test_engine_errors serve.test_control_cancel serve.test_parallel serve.test_restart_waiters serve.test_fatal_recovery -v` — 40 tests passed. `git diff HEAD^ HEAD --check` passed. The existing commit records a separate prior Linux run of 46 focused tests; its full suite retained six baseline missing-regex errors, so no full-suite pass is claimed. No GPU or real model was used.

Developed, tested and reviewed with OpenAI coding assistants.

Mehr auf der Site

Links zu Install, Modellen, Releases.