Pull requests / #748

#748 serve: invalidate the engine after fatal verification timeouts

closed · @zengyishou · 0 Kommentare · Auf GitHub

Server & APIWindows

Beschreibung

## Summary

A native verification timeout releases the verifier's GPU waits permanently, but the process can remain alive and emit a trailing `DONE`. The server currently treats the `ERR` as a recoverable request error and continues reporting `loaded: true`; subsequent requests can reuse an unusable verifier.

Recognize only the fatal verification-timeout signatures in the stdout pump, mark that process unavailable, terminate it, and publish EOF instead of the trailing `DONE`. The failed request returns an error without automatic replay; the existing lifecycle reloads the engine for the next request. Ordinary request validation errors remain recoverable. Reject generation on an already unavailable engine before accessing its stdin.

## Validation

Synthetic child-process regressions cover both timeout signatures, HTTP stream errors and health status, next-request reload, ordinary validation errors, and a closed engine. These tests exercise real process pipes without a GPU or private workloads. This change does not address the underlying GPU timeout or change watchdog limits.

Command: `python -m unittest discover -s serve -t .`

Results on macOS with Python 3.12: 202 tests run, 195 passed and 7 skipped. The five new regression tests pass; against the unchanged main branch they reproduce one failure and three errors. POSIX executable fixtures are skipped on Windows.

Mehr auf der Site

Links zu Install, Modellen, Releases.