Pull requests / #3556

#3556 [Engine] preserve streamed tool aliases and confirm wrapper boundaries

open · @Tsury · 0 コメント · GitHub で見る

Server & APIAMD / HIPNVIDIA / CUDALinux

本文

## Problem and result

A declared namespaced tool written with its flat alias or padded name can lose its schema while streaming. An ambiguous alias can also route to the first namespace. Canonicalize only unambiguous aliases at the parser boundary, with exact declared names taking precedence, so names and parameter types agree across streamed and completed calls.

This refresh also fixes the quoted incomplete-wrapper example identified in review. `Format: <tool_call>... </function> and then I explain it.` now stays text rather than becoming a completed call. A following actual call opener still separates concatenated calls without merging their arguments.

## Related discussion

Replaces #2284 because its requested `/reopen` check passed but GitHub refused reopening with HTTP 422 `Validation Failed` (empty errors). REST and GraphQL reopening also failed after restoring the original branch head. The bot explicitly recommended a new linked PR: https://github.com/Niko1221/Strata/pull/2284#issuecomment-6310183962

Review: https://github.com/Niko1221/Strata/pull/2284#issuecomment-6184667095

Tracker reset/reopen instructions: https://github.com/Niko1221/Strata/pull/2284#issuecomment-6272913330

## Approach

Keep alias resolution and schema lookup together in the shared parser, removing the redundant Responses-only alias map. Preserve native unknown names, literal markers and opt-in recovery batches.

Keep early streamed names and argument pieces, but hold their final JSON closing brace until the wrapper or next-call boundary confirms completion. Preserve native streaming recovery when output ends immediately after `</function>` on a natural stop (`None` or `stop`). Output limits, cancellation and disconnect keep an omitted-wrapper call incomplete; fully closed calls keep their existing behavior. Trailing prose cannot confirm an omitted wrapper closer. Responses marks an implicitly interrupted call incomplete; only the parser's completed call confirms it.

## Validation

**Passed:**

On Linux, Python 3.14.4, CPU-only mock engine, revision `13ca199865ae74bb1f7c40edc47867478cf20858`, rebased onto main `9de04cd44e927a9a3df6bbc0e94fc1be8b844ba4`:

```sh
python3 -m unittest serve.test_tool_call_unclosed serve.test_native_tool_parser serve.test_server.ToolCallRecovery serve.test_server.ToolCallRecoveryOff serve.test_server.UnfinishedToolCall serve.test_server.ToolCallTerminators serve.test_server.ToolCallTagInProse
python3 -m unittest serve.test_responses
```

75 tests passed across these two commands. Includes valid-call controls, byte/chunk boundaries, aliases and native schema types, concatenated calls, literal tags and quoted incomplete wrappers, and streamed/collected Chat Completions, Messages and Responses checks. `git diff --check` passed.

**Failed:**

Before this repair, both new HTTP regression methods failed against the original parser patch after rebase: Chat returned `tool_calls`, and Responses returned a completed function call. The two pure-parser literal scenarios also reproduced the unexpected call across all 24 tested streaming/recovery/chunk subcases. The added natural-stop boundary test and extended Chat/Messages max-token test also failed on the previous condition (10 subcase failures across two methods), then passed with this repair. Two combined-suite runs encountered the existing `NoLeakedSampler.test_server_close_ends_the_sampler` global thread-count timing failure (`1 != 2`). That class passed in isolation. Running the parser/API group and Responses module in separate processes passed all 75 methods, including that class. No test or production change was made for the unrelated thread-count failure.

**Not run:**

GPU inference, native client checks, or CUDA/HIP/SYCL builds. This contribution changes Python serving code only and makes no hardware or throughput claim.

## Limits

An incomplete wrapper can still announce a tentative streamed tool name and partial arguments. Those arguments remain incomplete JSON; no completed tool call follows. Chat and Messages collected replies omit that tentative call. Responses retains an explicitly incomplete item beside the literal text. Fully buffering all call events would remove early argument streaming and is outside this fix.

関連リンク

インストール・モデル・リリースへのサイト内リンク。