Pull requests / #762

#762 serve: extract JSON from prose and code fences

closed · @cipgysmo · 0 评论 · 在 GitHub 查看

BenchmarksServer & API

描述

In tool-heavy runs, the model often returns valid JSON after a short explanation or inside a Markdown code fence. `validated_json()` previously passed the whole assistant message to `json.loads()`, so those otherwise-correct structured responses failed.

This adds a small `_extract_json()` helper that:

- strips a leading Markdown code fence,
- finds the first balanced top-level `{...}` object,
- leaves non-object text unchanged so existing error messages remain useful.

This was one of the highest-value fixes in our local dual-GPU tool-eval runs: it made structured output tolerant of normal model prose without changing the benchmark or forcing extra JSON-only prompts.

站内延伸阅读

链到安装、模型与版本说明,便于 SEO/GEO,非官方 issue 正文。