Pull requests / #1678
#1678 serve: keep-alive while a tool call's held-back argument is generated (#1666)
open · @nekomario28 · 0 comments · View on GitHub
Description
Fixes #1666. (The patch is the reporter's, applied to main; this adds the ping-cadence unit test the issue offered.) With `stream_tools`, `OutputParser` holds an array/object argument until `</parameter>` — a large one (a measured 66 KB `edit_file` call) left the stream silent for minutes while tokens were generated, so a client's idle timeout ended a healthy request. Forge's is 120 s; the reporter hit it on a real agent turn. ## Change `Service.run`'s token loop now tracks `last_out` (set on the heartbeat ping and whenever the parser emits events). When a token produces no event and nothing has gone out for `KEEPALIVE_S` (10 s), it yields the same `"ping"` the engine-quiet heartbeat sends — no new event type; clients already ignore SSE comments. Ordinary answers are untouched: a token that streams content keeps resetting `last_out`. ## Test plan - New `HeldBackArgKeepalive` in `serve/test_server.py`: a fake engine whose generated `<tool_call>` is held whole by the frontend (the same silence a long array argument makes), over a patched clock — asserts a `ping` still goes out and the request reaches `done`; and a control that an ordinary answer emits no ping. - Full `serve.test_server`: 273 tests, 3 errors all `ModuleNotFoundError: regex` — the missing optional dep in this environment, identical on a clean checkout; unrelated. - Reporter's live reproduction: longest gap between SSE events 27.8 s → 10.0 s, the 7.6 KB array arriving intact; an 880 s Forge agent turn with a 66 KB argument kept alive. Not tested here: a real 120 s idle-timeout client (the reporter measured that end); the keep-alive itself is covered above.
Related on strata.com
Editorial links to help you install, pick models, or read release notes — not part of the upstream thread.