Pull requests / #351

#351 serve: preserve context across engine restart

closed · @updalla-apshir · 0 comments · View on GitHub

Server & APIModels & quants

Description

## Summary

During an engine restart, `max_context` and related model metadata temporarily become `0` while the replacement engine is loading.

This causes incorrect values across `/v1/status`, `/v1/models`, `/props`, `/health`, `/slots`, and `/metrics`.

This change preserves the outgoing engine's context and metadata until the replacement engine reports its own values, while keeping readiness detection independent.

## Testing

* Added a regression test covering context preservation across restart.
* `serve.test_server`: **68 tests passed**
* Verified the original implementation reports `0` and the fix preserves `131072`.
* Verified a replacement engine with a different context correctly updates the value.

This change only addresses the context/metadata reporting issue during restart.

Related on strata.com

Editorial links to help you install, pick models, or read release notes — not part of the upstream thread.