Pull requests / #790

#790 serve: honour OpenAI tool_choice "required" and a named function

closed · @aguindehi · 0 comentários · No GitHub

Server & APIModels & quants

Descrição

## What

`/v1/chat/completions` now honours a forced `tool_choice`:

- `"required"`: the reply is a call to one of the request's tools.
- `{"type": "function", "function": {"name": N}}`: the reply is a call to `N`.

Strata has no grammar decoder, so the server writes the call's opening itself:
`<tool_call>\n<function=` (plus `N>\n` for a named choice). The model can then
only continue with a call. Where the opening goes:

- thinking off: at the end of the prompt;
- thinking on: where the thinking ends, at a clean point (no tag held back, no
  character split across tokens), the same way the `reasoning_budget_tokens`
  wrap-up continues from the prefix the engine already holds;
- budget wrap-up: right after `REASONING_WRAP_UP`.

`"auto"` and no `tool_choice` are unchanged. `"none"` offers no tools, as the
Responses route already does. An unknown name, a forced choice without tools,
other values, and a forced choice together with MCP tools are a 400 instead of
being silently ignored.

## Why

Agents force a tool on specific turns. Hindsight's reflect agent, for example,
sends `tool_choice: "required"` on its first turns and on its `done` call.
Until now Strata ignored `tool_choice`, so those turns could come back as text.

## Limits

- Only the call is forced. The arguments are not constrained (the same level
  as `response_format` today).
- A reply cut by `max_tokens` stays `finish_reason: "length"` without
  `tool_calls`, as before.
- Chat Completions only. The Responses route and Anthropic `tool_choice` can
  reuse `forced_call()` in a follow-up.

## Tests

`serve/test_server.py` gets `ForcedToolChoice` (6 tests, with `MockEngine` and
`ByteTokenizer`): `"required"` and named, thinking on and off, streamed and
whole replies, budget wrap-up, `"auto"`/absent/`"none"`, a call cut by max
tokens, and the refused values. `python -m pytest serve` passes: 267 passed,
7 skipped.

The same change, on `v0.1.38` (`99f3dbd`), has been deployed behind LiteLLM
since 2026-10-03 for Hindsight's forced reflect turns.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_013Dd7NbnghR6cgQdQ2rTkUZ

No site

Links install, modelos, releases.