Pull requests / #2371
#2371 fix(serve): coerce XML tool arguments consistently by schema
open · @anstaendig · 0 comments · View on GitHub
BenchmarksServer & APIAMD / HIPNVIDIA / CUDA
Description
## Summary XML tool calls can contain Python-style literals such as `<parameter=unlock>False</parameter>`. JSON decoding fails and currently passes `"False"` to clients as a truthy string, which can reverse the intended action. Convert supported scalar literals using the declared type in both completed and streamed tool calls. Related to #1955, which covers boolean conversion in the completed-call parser. This change also applies conversion to streamed argument fragments and covers nullable and numeric parameters. ## What changed - Add one shared conversion helper for `parse_tool_call()` and `OutputParser._scan()`. Accept case-insensitive booleans, nullable `None`, and decimal numeric fallbacks when the declared type permits them. - Preserve string-only parameters verbatim, including `["string"]`. Preserve existing JSON-first decoding, including quoted JSON strings, objects and arrays. Unknown or incompatible values stay raw; new numeric fallbacks reject non-finite values. - Add parser regressions in `serve/test_tool_argument_types.py`, including whole output, character-wise streaming, every two-chunk split, multiple calls, literal tags inside strings and incomplete parameters. Add a localhost mock-engine test for streamed and non-streamed OpenAI and Anthropic responses. ## Extra Notes Validation on upstream `296b59d2` plus this change, macOS / Python 3.12, using cached Jinja2, regex and jsonschema dependencies: ```sh python -m unittest serve.test_tool_argument_types serve.test_frontend serve.test_server serve.test_responses serve.test_mcp -q ``` 412 tests passed. An additional 1,995 differential checks against upstream `296b59d2` passed for legacy JSON/string/raw values, reasoning/content events, literal tags, truncation and chunking. `git diff --check` passed. The tag delimiters and existing parser state transitions are unchanged. No C++/CUDA/HIP/SYCL backend files are touched. No deployment, GPU inference or BFCL rerun was performed; benchmark recovery is unmeasured.
Related on strata.com
Editorial links to help you install, pick models, or read release notes — not part of the upstream thread.