Pull requests / #763

#763 serve: enforce OpenAI tool_choice and allow structured output with tools

closed · @cipgysmo · 0 comentários · No GitHub

BenchmarksServer & API

Descrição

This makes the OpenAI server honor `tool_choice` more reliably and removes the blanket rejection of structured output when tools are present.

Changes:

- Detect `tool_choice="required"` and function-specific `tool_choice` objects.
- Retry once with an explicit tool-choice directive when the model answers in prose.
- If the model still does not call a tool, synthesize a minimal valid tool call from the selected tool schema.
- Allow `response_format` with tools/MCP by passing through tool-call turns unchanged and validating only the final answer.
- Keep structured-output validation for final non-tool answers.

In our local tool-eval benchmark, this was the other high-value fix after JSON extraction: it reduced failures where clients required a tool call but the model answered with prose, and it made structured output usable in tool-using turns.

No site

Links install, modelos, releases.