Pull requests / #2952

#2952 fix(serve): order chat tool results by tool_call_id

open · @j-luwierski · 0 コメント · GitHub で見る

Server & APIAMD / HIPNVIDIA / CUDALinux

本文

## Summary

Chat Completions history conversion drops assistant tool-call IDs and tool-result `tool_call_id` values. If an assistant calls `read_file(a.txt)` followed by `read_file(b.txt)` and the client sends results B then A, the template receives two anonymous `<tool_response>` blocks in B/A order. Their association with the calls is lost. Responses already handles call-ID ordering independently.

Preserve IDs during Chat Completions conversion and reorder each complete result run to the preceding assistant turn's call order before removing the IDs. This also handles multiple calls of the same function, where matching by name cannot help.

## What changed

- `serve/frontend.py`: retain temporary call IDs and normalize result order only when the calls and results have matching sets of unique, nonempty string IDs and equal counts.
- `serve/test_chat_tool_results.py`: add seven tests for reversed same-function results, unchanged ordered/legacy prompts, missing/duplicate/unknown/malformed IDs, incomplete histories, separate turns, JSON-encoded history, image-bearing results, input immutability and HTTP-to-engine ordering.

Missing, ambiguous or incomplete IDs keep the received order; results never move across an intervening message. Private IDs are removed before rendering. No request or response fields are added.

Anthropic conversion, Responses ordering, templates, sampling, engine code and defaults are unchanged. The existing Responses helper is left alone so its treatment of incomplete histories does not change.

## Validation

Linux, Python 3.13, based on upstream `477ac13cbe8e582c20484871ce459d20d82df09d`:

- New test module against the original frontend: seven tests run, four fail (including the HTTP engine-prompt check). All seven pass after the fix.
- `python -m unittest serve.test_chat_tool_results serve.test_server serve.test_responses -q`: **384 tests pass, 1 skipped**, 110.4 s. Uses the repository's existing Python environment.
- Compared original and changed conversion results and rendered UTF-8 prompt bytes for 17 ordered, legacy, single-call and ambiguous histories x thinking on/off x list/JSON-string inputs: **68/68 identical**. Only complete out-of-order result runs are intentionally changed.
- `git diff --check` passes.

Tests use the real chat template and a mock engine. No GPU/model inference or model-quality measurement was performed; no CUDA/HIP/SYCL source or build files are touched, and backend builds were not run.

## Related work

Searched open issues and PRs for `tool_call_id`, tool results and out-of-order handling; no matching Chat Completions fix found.

関連リンク

インストール・モデル・リリースへの站内リンク。