Pull requests / #1183

#1183 serve: release batch control after terminal native ERR

closed · @hulkbig · 0 comments · View on GitHub

Server & APILinux

Description

Fixes #1059

A native validation rejection emits `ERR` and returns to the input loop without `DONE` or `BADM`. With batch control active, the Python generator's cleanup then sends STOP and waits up to 300 seconds for a terminal line that will never arrive, holding the control lock and delaying the next request.

Raise a dedicated `EngineRequestError` (still a `ValueError`) for an ERR consumed by `_control`, and mark that command terminal before cleanup. Malformed protocol values still drain; cancellation, consumer close, fatal errors and restart cleanup retain their existing behavior. The production drain timeout is unchanged.

Regression tests use real HTTP and pipes with a scripted native process. A rejection is gated until a second request is queued; both requests must finish promptly, release the control lock and reserved slot, and preserve another active slot. They also exercise the existing malformed-protocol, consumer-close and fatal-error cleanup paths.

Validation on macOS arm64 / Python 3.12.12: `python -m unittest serve.test_engine_errors serve.test_control_cancel serve.test_parallel serve.test_restart_waiters serve.test_fatal_recovery -v` — 40 tests passed. `git diff HEAD^ HEAD --check` passed. The existing commit records a separate prior Linux run of 46 focused tests; its full suite retained six baseline missing-regex errors, so no full-suite pass is claimed. No GPU or real model was used.

Developed, tested and reviewed with OpenAI coding assistants.

Related on strata.com

Editorial links to help you install, pick models, or read release notes — not part of the upstream thread.