#6474 [Engine] serve: keep a restarted engine's batch slot state out of old request cleanup
open · @larryllr · 0 commentaires · Sur GitHub
Server & APIAMD / HIPNVIDIA / CUDAWindowsLinux
Description
## Problem and result
Two cleanup paths in `terminal/serve/server.py` still share the restart generation race fixed in `_release_slot_when_done()` (#4451 / #5657):
1. In `StrataEngine.generate_batched()`'s `finally:` block, `same = self.gen == born` is evaluated once at the top, before blocking calls (`_drain_control()` up to `DRAIN_TIMEOUT_S`, `_bdone_after_eos()` up to 2 s). If the engine dies and `restart()` advances `self.gen` and allocates new `slot_busy`, `slot_claims`, `slot_held`, `slot_used`, and `slot_live` tables during that wait, the stale `same == True` causes the old request's cleanup to clear `self.slot_live[slot]`, `self.slot_held[slot]`, `self.slot_busy[slot] = False`, and `self.slot_claims.discard(slot)` (and the same for `reserved`) on the **new** engine's active slots — freeing a slot owned by a live request (or raising `IndexError` if the restarted engine has fewer batch slots).
2. In `StrataEngine._quarantine_control()`, `self.slot_releasing`, `self.slot_claims`, and `self.slot_busy` are mutated without capturing the old generation's slot tables before checking `self.gen == born` (and inside `run()`, `self.gen == born` is checked before `self.alive()` and before acquiring `slot_cv`), so a restart right before entry or during quarantine can mutate the new engine's slot state.
After this fix, both paths follow `_release_slot_when_done()`'s invariant (`restart()` advances `self.gen` before replacing the per-process slot objects): they capture the slot lists/sets first, then check `self.gen == born` (after any blocking drains/waits) and bounds-check the slot index, mutating only the captured old-generation objects.
## Related discussion
Follow-up to the restart slot-table capture pattern in #4451 and #5657 (whose limits noted remaining restart/field-capture timing windows). Searched open and closed issues/PRs for `#5657`, `#4451`, `quarantine`, and `BADM`; none covers these two paths.
## Approach
- In `terminal/serve/server.py`:
- `StrataEngine._quarantine_control()`: capture `busy, releasing, claims = self.slot_busy, self.slot_releasing, self.slot_claims` before checking `if self.gen != born or slot >= len(busy): slot = None`, and mutate those captured objects both on entry and inside `run()`.
- `StrataEngine.generate_batched()` `finally:`: check `self.gen == born` freshly before `_bdone_after_eos()`, then after all drains/waits capture `busy, claims, held, used, live_slots` before evaluating `same = self.gen == born` and bounds-checking `reserved`/`slot` against `len(busy)`.
- In `terminal/serve/test_parallel_yield.py` (`ParallelServiceYield`):
- Add two deterministic, sleep-free regression tests modeled on `test_a_restart_during_the_release_keeps_the_new_engines_claim`:
- `test_a_restart_during_admit_drain_keeps_the_new_engines_slot`
- `test_a_restart_during_quarantine_keeps_the_new_engines_slot` (covers both pre-entry restart and mid-quarantine restart after `self.gen == born` is checked).
## Validation
**Passed:**
- Revision `f18f89a` (based on `origin/main` `3142b66`), Windows 11, Python 3.12 (`PYTHONPATH=terminal`):
- Before fix (both new tests against unpatched `3142b66`): failed as expected (`Ran 2 tests in 1.612s — FAILED (failures=2)` with `AssertionError: False is not true : the old request's finally freed the new engine's active slot` and `AssertionError: 0 not found in set() : pre-entry restart: quarantine erased the new engine's claim`).
- `python -m unittest serve.test_parallel_yield.ParallelServiceYield.test_a_restart_during_admit_drain_keeps_the_new_engines_slot serve.test_parallel_yield.ParallelServiceYield.test_a_restart_during_quarantine_keeps_the_new_engines_slot -v`: `Ran 2 tests in 1.644s — OK`.
- `python -m unittest serve.test_parallel serve.test_parallel_yield serve.test_admit_order serve.test_control_notify serve.test_stream_ownership serve.test_restart_waiters serve.test_quiet_cancel -v`: `Ran 123 tests in 393.662s — OK (skipped=1)` (1 skipped: Windows lacks `signal.SIGSTOP` for `test_an_undrainable_replica_is_not_handed_to_the_next_request`).
- `git diff --check`: clean (exit 0).
**Failed:**
- none observed.
**Not run:**
- Native engine/GPU builds (CUDA, HIP, SYCL, Metal) and Linux-only `SIGSTOP` test: Python `terminal/serve/` change only; Linux CI checks (`check.yml`, `terminal-minimal-server.yml`) cover POSIX execution.
## Limits
This addresses restart generation isolation in `generate_batched()`'s `finally:` cleanup and `_quarantine_control()`. No native engine C++ code was modified.Sur le site
Liens install, modèles, releases.