Pull requests / #748
#748 serve: invalidate the engine after fatal verification timeouts
closed · @zengyishou · 0 コメント · GitHub で見る
本文
## Summary A native verification timeout releases the verifier's GPU waits permanently, but the process can remain alive and emit a trailing `DONE`. The server currently treats the `ERR` as a recoverable request error and continues reporting `loaded: true`; subsequent requests can reuse an unusable verifier. Recognize only the fatal verification-timeout signatures in the stdout pump, mark that process unavailable, terminate it, and publish EOF instead of the trailing `DONE`. The failed request returns an error without automatic replay; the existing lifecycle reloads the engine for the next request. Ordinary request validation errors remain recoverable. Reject generation on an already unavailable engine before accessing its stdin. ## Validation Synthetic child-process regressions cover both timeout signatures, HTTP stream errors and health status, next-request reload, ordinary validation errors, and a closed engine. These tests exercise real process pipes without a GPU or private workloads. This change does not address the underlying GPU timeout or change watchdog limits. Command: `python -m unittest discover -s serve -t .` Results on macOS with Python 3.12: 202 tests run, 195 passed and 7 skipped. The five new regression tests pass; against the unchanged main branch they reproduce one failure and three errors. POSIX executable fixtures are skipped on Windows.
関連リンク
インストール・モデル・リリースへの站内リンク。