Pull requests / #231
#231 fix(serve): report a tool call cut off by the end of the output as unfinished
closed · @alphastorm · 0 评论 · 在 GitHub 查看
描述
Fixes #211 (the second trigger of #210: the output ends inside a call). When the model's output ended inside a tool call whose name had already streamed, `OutputParser.finish()` closed its JSON with `_close_scan()` and emitted `tool_call`, and the answer ended with `finish_reason: "tool_calls"` / `stop_reason: "tool_use"`. A client could not tell the call was cut and ran it with truncated arguments. In the web app's chat, an unfinished MCP call ran with empty arguments. **Change** - `frontend.py`, `finish()`: an announced call that never reached `</function>` stays unfinished. Its JSON is not closed and no `tool_call` follows its `tool_start`. A call missing only its `</tool_call>` is still whole. - `server.py`: the OpenAI and Anthropic streams say `"tool_calls"` / `"tool_use"` only when every streamed call got its `tool_call`; otherwise the engine's reason stands (`"stop"` / `"end_turn"`). `anthropic_collect` leaves out a `tool_use` block whose input never closed, since it has no input to give. The MCP loop needs no change: the call is not run, and the web app already shows a started call as "Not run" when the answer ends. - A cut at the token limit also keeps its arguments open now. Its finish reason is still `"length"` / `"max_tokens"`; the non-streamed Anthropic answer leaves out the cut `tool_use` block, where it used to carry the truncated input. **Tests** (`python -m unittest serve.test_server serve.test_mcp serve.test_detok`) - `UnfinishedToolCall`: the parser (fed 1, 7 and all characters at a time), and the four API shapes for a cut call and, as a control, a whole call. - `ToolLoop.test_a_call_the_output_ends_inside_is_not_run`. On `bbaaabb` (0.1.28) the new tests fail (5 failures: the cut-call cases and the MCP case; the whole-call control passes). With the change the suite passes (72 tests, 3 skipped) on Python 3.13.15 and 3.14.7, macOS arm64 with the mock engine. No GPU or real model was run, and the web app was not opened.
站内延伸阅读
链到安装、模型与版本说明,便于 SEO/GEO,非官方 issue 正文。