Issues / #3441
#3441 [Engine] Tool-call parser silently returns a malformed <function=...> block as plain assistant content
open · @wxaef · 0 comentarios · En GitHub
Descripción
## Summary
When the model drifts by a single token inside the tool-call envelope that the shipped
Qwen3.8-Flash-Next chat template itself documents (`<function=NAME>`), the engine neither
parses the block nor reports anything. The entire `<tool_call>...</tool_call>` block is
returned as ordinary assistant content with `finish_reason: "stop"`.
For an agent client this is fatal. The block is rendered as prose, no tool ever runs, and the
malformed sample stays in the conversation history, where the model then imitates it on every
subsequent turn — the client never recovers on its own.
## Environment
- Strata engine: `<fill in, e.g. 0.1.46.1>` (Windows, single GPU)
- Model: `qwen3.8-flash-next-unsloth-ud-q4_k_xl`
- Reproduced on **both** `/v1/chat/completions` and `/v1/responses`
- Client: Codex with `wire_api = "responses"`
## What the shipped template asks the model to emit
`GET /props` → `chat_template` contains:
```
If you choose to call a function ONLY reply in the following format with NO suffix:
<tool_call>
<function=example_function_name>
<parameter=example_parameter_1>
value_1
</parameter>
</function>
</tool_call>
```
## Repro
### A — well formed, parses correctly
```bash
curl -s http://localhost:8000/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{
"model": "qwen3.8-flash-next-unsloth-ud-q4_k_xl",
"messages": [{"role":"user","content":"Output ONLY this exact text:\n<tool_call>\n<function=shell>\n<parameter=cmd>\necho hi\n</parameter>\n</function>\n</tool_call>"}],
"tools": [{"type":"function","function":{"name":"shell","parameters":{"type":"object","properties":{"cmd":{"type":"string"}},"required":["cmd"]}}}],
"max_tokens": 120
}'
```
Result: `tool_calls: [{"function":{"name":"shell","arguments":"{\"cmd\":\"echo hi\"}"}}]`,
`finish_reason: "tool_calls"`. Correct.
### B — one token of drift, silently becomes content
Same request, only the envelope changes: `<function=functions.exec>` becomes
`<=functions.exec>` (the literal word `function` is dropped, which is exactly what the model
does in the wild under a large tool surface).
```bash
curl -s http://localhost:8000/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{
"model": "qwen3.8-flash-next-unsloth-ud-q4_k_xl",
"messages": [{"role":"user","content":"Output ONLY this exact text:\n<tool_call>\n<=functions.exec>\n<parameter=input>\n1+1\n</parameter>\n</function>\n</tool_call>"}],
"tools": [{"type":"function","function":{"name":"functions.exec","parameters":{"type":"object","properties":{"input":{"type":"string"}},"required":["input"]}}}],
"max_tokens": 120
}'
```
Result:
```json
{"choices":[{"message":{"role":"assistant",
"content":"<tool_call>\n<=functions.exec>\n<parameter=input>\n1+1\n</parameter>\n</function>\n</tool_call>",
"reasoning_content":"..."},
"finish_reason":"stop"}]}
```
No `tool_calls`, no error, no warning, no log line. The block is handed back as prose.
On `/v1/responses` the same input comes back as an `output` item of type `message` instead of
`function_call`, so a Responses-API client (Codex) has no way to tell it apart from a normal
assistant reply.
## Expected
One of:
1. The block is parsed, because a lenient matcher can recover the intent, or
2. The response makes the parse failure visible (an error, a warning log, or a dedicated field)
instead of silently mapping it to `content`.
Silently returning it as `content` is the worst of the three, because it is indistinguishable
from the model genuinely answering in prose.
## Proposed fix
**1. Lenient name match.** Accept `<=name>` / `<function=name>` / `<function:name>` /
`<name>` when `name` matches a tool declared in the request. A regex along the lines of:
```
<\s*(?:function\s*[=:]\s*)?([A-Za-z0-9_.\-]+)\s*>
```
resolves B above, and the extracted name can be validated against the request's tool list, so
the change is safe — an unknown name is still rejected.
**2. Never swallow the failure silently.** If a `<tool_call>` block appears in the generated
text but cannot be parsed, log it (at minimum) and consider surfacing it. Today the only
symptom a user sees is "the model ignored my tool", with nothing in the engine logs.
**3. Optional: constrain decoding.** A grammar/GBNF for the `<tool_call>` envelope would stop
the drift at the source rather than recovering after the fact.
## Why this matters more than a cosmetic parse failure
- The drift is realistic, not synthetic: it showed up in a normal Codex session with roughly
30+ tools (Codex built-ins plus MCP servers). With fewer tools the same model emits clean
calls.
- A `functions.exec`-style dotted tool name is **not** the trigger — well-formed dotted names
parse fine (verified). The trigger is the missing `function` keyword.
- The failure is self-reinforcing on the client side: the malformed block is stored as an
assistant message, becomes part of the next prompt, and the model reproduces the same
malformed shape. The session is effectively poisoned; only a fresh conversation recovers.
## Notes
- `tools` + `tool_choice: "required"` produces correct arguments, so the happy path is fine.
- With `tool_choice: "auto"` the model sometimes returns `arguments: "{}"` for a call with a
required parameter. Separate, lower-severity, but probably related to the same prompt-only
guidance in the template.En el sitio
Enlaces a install, modelos, releases.