Pull requests / #1172

#1172 serve: opt-in recovery of tool calls the model writes next to the template's form ("tool_call_recovery": true)

closed · @signalnine · 0 comments · View on GitHub

Server & APIDocumentation

Description

Replaces #966, which the history rewrite closed. Per your note there, it's opt-in now: `"tool_call_recovery": true` in `strata-<model>.json`. Without the key every reply comes back exactly as on `main`.

With it on, a call of a tool the request declared is the call when the model writes it as:

| the model wrote | `main` | with `tool_call_recovery` |
| --- | --- | --- |
| `<tool_call><parameter=NAME>...` (the tool's name as a parameter tag) | text | the call |
| `<tool_call>{"name": NAME, "arguments": {...}}</tool_call>` (also `"parameters"`, or arguments as a JSON string) | text | the call |
| `<function=NAME>...</function>` at the start of a line, no `<tool_call>` (a stray `</tool_call>` after it is dropped) | text | the call |
| two calls inside one `<tool_call>` (the second may open as `<parameter=NAME>`) | ONE call, the second's parameters merged into the first | two calls |

Anything else in those forms comes back as the text it is: an undeclared name, broken JSON, a JSON object that never closes. A bare `<function=` only opens a call at the start of a line and outside code. That check uses the parser's own fence/inline-code tracking from #804/#1058, so it follows the same rules as a call inside the reasoning.

The last row is the one I'd flag even with the switch off: on `main` a batch in one wrapper silently becomes one call with mixed arguments. I kept the fix behind the switch as asked, but it might deserve to be the default on its own.

## Measured

A corpus of 1,462 agent turns in 176 shapes, captured from Qwen3.8-27B under Claude Code (redacted, from [signalnine/q27](https://github.com/signalnine/q27)'s drift corpus), each labelled with the calls the model meant, replayed through `OutputParser` on v0.1.40:

| | turns read as intended | shapes | turns that end the request |
| --- | ---: | ---: | ---: |
| `main` (v0.1.40) | 92.9% | 120/176 | 0 |
| `tool_call_recovery: true` | 98.3% | 154/176 | 0 |

Streamed output equals whole output at 1- and 7-character chunks, with and without `stream_tools`. The 25 turns still missed are mostly invalid JSON and redaction artifacts. @blange48 ported the earlier version onto v0.1.40 and runs it; this one should drop in the same way.

## Tests

- `ToolCallRecovery` (`serve/test_server.py`): every recovered form and every form that must stay text, at steps 1 / 7 / whole with `stream_tools` on and off. Announced calls' streamed arguments equal their final ones.
- `ToolCallRecoveryOff`: without the switch the same inputs give exactly what `main` gives, the batch merge included.
- `ToolCallRecoverySwitch`: the config key reaches the parser through `Service.run`.
- `python -m unittest discover -s serve -p 'test_*.py' -t .`: 464 tests OK (452 on `main`).
- `docs/DETAILS.md`: a paragraph after Streaming.

Related on strata.com

Editorial links to help you install, pick models, or read release notes — not part of the upstream thread.