Pull requests / #231
#231 fix(serve): report a tool call cut off by the end of the output as unfinished
closed · @alphastorm · 0 Kommentare · Auf GitHub
Beschreibung
Fixes #211 (the second trigger of #210: the output ends inside a call). When the model's output ended inside a tool call whose name had already streamed, `OutputParser.finish()` closed its JSON with `_close_scan()` and emitted `tool_call`, and the answer ended with `finish_reason: "tool_calls"` / `stop_reason: "tool_use"`. A client could not tell the call was cut and ran it with truncated arguments. In the web app's chat, an unfinished MCP call ran with empty arguments. **Change** - `frontend.py`, `finish()`: an announced call that never reached `</function>` stays unfinished. Its JSON is not closed and no `tool_call` follows its `tool_start`. A call missing only its `</tool_call>` is still whole. - `server.py`: the OpenAI and Anthropic streams say `"tool_calls"` / `"tool_use"` only when every streamed call got its `tool_call`; otherwise the engine's reason stands (`"stop"` / `"end_turn"`). `anthropic_collect` leaves out a `tool_use` block whose input never closed, since it has no input to give. The MCP loop needs no change: the call is not run, and the web app already shows a started call as "Not run" when the answer ends. - A cut at the token limit also keeps its arguments open now. Its finish reason is still `"length"` / `"max_tokens"`; the non-streamed Anthropic answer leaves out the cut `tool_use` block, where it used to carry the truncated input. **Tests** (`python -m unittest serve.test_server serve.test_mcp serve.test_detok`) - `UnfinishedToolCall`: the parser (fed 1, 7 and all characters at a time), and the four API shapes for a cut call and, as a control, a whole call. - `ToolLoop.test_a_call_the_output_ends_inside_is_not_run`. On `bbaaabb` (0.1.28) the new tests fail (5 failures: the cut-call cases and the MCP case; the whole-call control passes). With the change the suite passes (72 tests, 3 skipped) on Python 3.13.15 and 3.14.7, macOS arm64 with the mock engine. No GPU or real model was run, and the web app was not opened.
Mehr auf der Site
Links zu Install, Modellen, Releases.