Pull requests / #748
#748 serve: invalidate the engine after fatal verification timeouts
closed · @zengyishou · 0 comments · View on GitHub
Description
## Summary A native verification timeout releases the verifier's GPU waits permanently, but the process can remain alive and emit a trailing `DONE`. The server currently treats the `ERR` as a recoverable request error and continues reporting `loaded: true`; subsequent requests can reuse an unusable verifier. Recognize only the fatal verification-timeout signatures in the stdout pump, mark that process unavailable, terminate it, and publish EOF instead of the trailing `DONE`. The failed request returns an error without automatic replay; the existing lifecycle reloads the engine for the next request. Ordinary request validation errors remain recoverable. Reject generation on an already unavailable engine before accessing its stdin. ## Validation Synthetic child-process regressions cover both timeout signatures, HTTP stream errors and health status, next-request reload, ordinary validation errors, and a closed engine. These tests exercise real process pipes without a GPU or private workloads. This change does not address the underlying GPU timeout or change watchdog limits. Command: `python -m unittest discover -s serve -t .` Results on macOS with Python 3.12: 202 tests run, 195 passed and 7 skipped. The five new regression tests pass; against the unchanged main branch they reproduce one failure and three errors. POSIX executable fixtures are skipped on Windows.
Related on strata.com
Editorial links to help you install, pick models, or read release notes — not part of the upstream thread.