#4306 [Other] serve: session ladder — conversations survive an engine unload
open · @sakuumomo · 0 comentários · No GitHub
BenchmarksSetup & installServer & APIAMD / HIPNVIDIA / CUDAModels & quantsDocumentation
Descrição
Problem and result: Before: a conversation held in the engine died when the engine unloaded (manual click, idle unload, or crash) or when another client took the active slot — the next request re-prefilled from cold, about 2 minutes for a 145k-token agent conversation on a desktop. After: an opt-in session ladder checkpoints conversations to disk (switch-away saves at turn end, save-all before every unload, idle saves), hands the checkpoint back to the engine before the next request (warm restore), and the same prompt reads from the restored prefix in ~3 s at full reuse. Absent the `session_ladder` config block, every code path stays exactly as before. A follow-up on this branch closes a subagent hole found on a live host: OpenCode stamps every child (subagent) request with its own id in `x-opencode-session-id`, while the affinity family (`X-Session-Id`, `x-session-affinity`, `x-opencode-session`) carries the PARENT session id — the client assembles it as `parentID ?? fork source ?? own id` for provider-side cache affinity. With affinity-first extraction every subagent collapsed into the main session: never tracked, never saved to disk, its turn figures misattributed to the parent (the engine log named only main session ids across days of subagent use). The requesting session's own id now outranks the family, and a subagent earns its own checkpoint file and its own ledger entry.
Dependencies: none — this branch is `main` + 16 commits (re-based state; see the 2026-11-20 changes section) and lands on its own. PR #4307 stacks on top of this branch.
Related discussion: follow-up PR #4307 ("unload that works") makes the unload controls ride on these checkpoints.
Approach: a ladder of ledger + policy + plumbing around the engine's own session-file SAVE/RESTORE. Save-on-switch keeps a multi-client host (agent + webui + API clients) from losing context to slot churn. `POST /unload` checkpoints every dirty conversation before the engine closes. Warm restore re-attaches the checkpoint during the engine start, before the next request's first token. Refused saves/restores (park budget, disk) degrade to a normal cold prefill with one log line and never fail a request. A child session's parent (recorded from `x-opencode-parent-session-id` / `x-parent-session-id`) is kept only as a retention mark, never as identity. Subagent checkpoints are retained on their own terms: an over-cap save retires an orphan or child file before an older main file (a child re-prefills a subagent conversation, a main re-prefills its whole multi-GB context); `retention_max_child_files` caps child files so a fleet never occupies the whole ledger; a clean child checkpoint no request has touched for `child_ttl_s` is retired on the idle beat — a finished subagent never returns to the engine, and silence is the only completion signal the wire carries. A main file is never swept, a dirty child is never swept (its save is still owed), and the current session never loses its safety-net file. Config: `session_ladder` block (`idle_save_s` 8.0, `retention_max_files` 16, `retention_max_child_files` 8, `child_ttl_s` 21600, 0 = the sweep off); the park budget is read from the engine's own `--conversation-cache-mib`, and an enabled ladder without a resolvable budget refuses to start rather than coerce it. Includes the dormancy fixes: fidelity matches a grown single message, and a zero-reuse DONE unparks a stale park claim (the live incident this branch self-heals from). `docs/DETAILS.md` carries the user-facing paragraph.
Validation:
Passed: the full serve suite is green on trees containing these commits — the owner's deploy branch `nixos` rebuilt on upstream `v0.2.8` (1386 passed, 17 skipped; see Failed). The subagent rules add nine pins: parent-header extraction, child-before-older-main retirement, the separate child cap, the TTL sweep with its guards (main never swept, dirty child never swept, current session never swept, 0 disables), a child's mark surviving a restart through the ledger sidecar, and a live-HTTP child request landing as its own session with its parent recorded. Measured live on momo (RX 6900 XT, 262144 ctx, 2026-10-25..27): warm restores brought back 144,590 / 152,129 tokens with the 145k–152k prompt read completing in 3 s (cold prefill is ~2 min at ~1200 tok/s on this box); a field incident of a parked session silently going cold was reproduced deterministically, fixed, and re-verified on the deployed tree.
Failed: two `serve/test_lifeline.py` watcher tests (the engine-subprocess watchdog) error intermittently under heavy concurrent load on this machine and pass in isolation on the unmodified base — environment-sensitive, pre-existing, unrelated to this PR.
Not run: serve-side only, so no engine builds are touched; hybrid hosts in the owner's fleet run with the ladder deliberately off (config rendering verified, no multi-device park measurement).
Limits: opt-in only; park files grow with the conversation (~1.2 GB per 63k tokens, stated in the startup log line) under a 16-file retention cap with an 8-file child sub-cap; restore only accepts files written by the same server.
## Changes since the last update (2026-11-07)
- Rebased onto upstream `v0.2.12` (`123bad4a`, engine 0.1.58); branch tip `64925fb4` (still 13 commits). One request-path conflict, resolved as a fusion: the ladder's token-accounting hook now sits inside upstream's (#471/#3040) `seen = 0 if admission_rejected else prompt_tokens_seen(...)` shape, so a rejected admission still counts nothing.
- Gate refresh on the rebased trees: the full serve suite runs 1640 tests OK / 37 skipped on the deploy lineage carrying this branch (upstream's suite grew since the 0.2.8 wave), and the ladder trio (`test_session_ladder`, `test_session_ladder_lifecycle`, `test_session_ladder_wiring`) is 123 tests OK, all subagent pins intact. The owner's flavor builds (HIP gfx1030, CUDA+HIP hybrid gfx1150) are green with installCheck on the redeployed train `nixos` @ `c3e4cdc4`. The runtime-requirements and title jobs are green on the new head; the PR is mergeable against current `main`.
- Ladder behavior is unchanged: opt-in, same config block, same park-budget rule.
Commits were re-signed (GPG ed25519) on 2026-11-07 after rebasing onto upstream `v0.2.12` (`123bad4a`); branch tip `64925fb4`; the pre-rebase tip `0849fb78` stays fetchable as an ancestor of the train tag `pin-f6f00908`.
## Changes since the last update (2026-11-20)
- Rebased onto upstream `v0.2.17` (`12d491da4`); branch tip `43a52ce9` (16 commits: the thirteen re-landed plus adaptations your tree's own movement required — among them the ladder unit-CFG hygiene fix moved here from the #4551 chain, now this branch's tip: the unit config points at a fixture slot path instead of a live session directory). Three fusions where your tree moved under us, all keeping your behavior intact: the refusal-class registration now lives in `serve/server_parts/engines.py` next to `SessionRefused` itself (your file split, `7d77f51ca`), so one registration covers both entry modes (imported `serve.server` and `server.py` as `__main__`); `ConvCacheLog` keeps your #6081 refusal counters (`SKIP`, `REASONS`, `/metrics`) exactly as you wrote them, with the ladder's `drain_events()` intake parsed from the same poll loop — and the ladder's own card test now pins those counters rather than the pre-#6081 shape; and the request-path hook reads `svc.session_ladder` with `getattr`, because your `test_internal_error_500` (added after 0.2.12) builds `_post` on a bare service and the ladder must not require attributes the handler contract does not provide.
- Gate refresh on the rebased tree: the ladder trio (`test_session_ladder`, `test_session_ladder_lifecycle`, `test_session_ladder_wiring`) is 123 tests OK with every subagent pin intact; the full serve suite runs 2209 tests with only two non-green entries — `test_reader_choice` (telemetry_adl) and one lifeline child-process check — environment-sensitive, pre-existing, failing identically on a clean `v0.2.17` checkout on this machine and unrelated to this PR. The engine batteries on the same lineage are green (#3551's 2026-11-20 update).
- The owner's desktop host was switched to the new pin on 2026-11-20 (train `nixos` @ `4cfc06018`): the live engine's startup log shows slot save/restore on and the ladder's idle-save beat armed. Ladder behavior is unchanged from the validated design: opt-in, same config block, same park-budget rule.
- No CI checks were reported on the new head; the PR is mergeable against current `main`.
Commits were re-signed (GPG ed25519) on 2026-11-20 after rebasing onto `v0.2.17`; trees are byte-identical to the gated unsigned tips (tags `gate0217-*`); the pre-rebase tip `64925fb4` stays fetchable as an ancestor of the tag `pin-pre0217-pr-serve-session-ladder`.
## Changes since the last update (2026-11-21)
Rebased onto upstream `v0.2.18` (`6e38f588a`); branch tip `43a52ce9` -> `fc2030346`. 1:1 range-diff against the 0.2.17-era chain: the same sixteen commits, zero conflicts, zero adaptations — the upstream movement in `terminal/serve/server.py` (hy4 engine plumbing) and the replayed ladder hunks coexisted without touching each other. Gates re-run on the rebuilt lineage `nixos` @ `295872912` (88 signed commits): the ladder trio 123/123 (`test_session_ladder` 51, `test_session_ladder_lifecycle` 38, `test_session_ladder_wiring` 34), `test_server` 448 OK, and the full serve suite at 2249 tests (upstream's hy4 dialect/e2e tests joined) with 178 of 181 files green. The three non-green files are upstream-`v0.2.18` artifacts, reproduced identically on a pure `v0.2.18` checkout in the same pinned environment: `test_stop_closed_pipe` (upstream `server.py` now calls `sampling_keys(..., hy4=...)` while that test's stub lambda predates the kwarg), `test_mcp_image_default` (its source-pin still searches `svc.vision is not None` where upstream moved to `svc.has_vision()`), and `test_telemetry_adl::test_reader_choice` (the known environment-sensitive entry from the 2026-11-20 note); none of the three test files is touched by this stack. The CPU session family 7/7, ThreadSanitizer build, and the HIP gfx1030 battery 20/20 re-ran green too. The owner's desktop host was re-pinned to this lineage on 2026-11-21 and runs the session ladder live. Commits re-signed (GPG ed25519); trees unchanged from the validated work. The pre-rebase tip `43a52ce95` stays fetchable as an ancestor of the tag `pin-4cfc06018`. No CI checks were reported on the new head; the PR is mergeable against current `main`.
No site
Links install, modelos, releases.