Pull requests / #2140
#2140 serve: honour llama-server's thinking_budget_tokens (a llama.cpp client's thinking budget was silently ignored)
open · @xrip · 0 comentários · No GitHub
Server & APIMulti-GPUModels & quantsDocumentationWindows
Descrição
Fixes #2141.
## The bug
A client written for llama-server sends its thinking budget as `thinking_budget_tokens`. Strata's server only read
`reasoning_budget_tokens`, so it ignored that field: **the request ran with no thinking budget at all**, and nothing
was logged or reported.
llama-server reads both names. `tools/server/server-common.cpp` takes `reasoning_budget_tokens` and falls back to
`thinking_budget_tokens`:
```cpp
int reasoning_budget = json_value(body, "reasoning_budget_tokens",
json_value(body, "thinking_budget_tokens", -1));
```
Its Anthropic adapter (`server-chat.cpp`) also writes the second name. So any client built against llama-server's
behaviour can hit this when it is pointed at Strata.
## How it showed up
**Client:** the pi coding agent with its llama.cpp provider ([pi-llama-cpp](https://github.com/gsanhueza/pi-llama-cpp)),
on 2x CMP 50HX with Qwen3.8-Flash-Next IQ3_XXS.
**What the provider sends:** it maps pi's thinking level to a budget and sends `thinking_budget_tokens`. The `medium`
level, pi's default, is 8,192 tokens.
**What happened during a compaction:** pi's automatic compaction is a summarization request over the serialized history,
109,013 tokens here. pi caps that answer at 0.8 x `reserveTokens`, which was 26,214 tokens.
- Without the budget, the model kept thinking about the history until it reached that cap. It never got to the summary.
- pi received no summary and sent the same request again: **five times in a row, about 250 s each** (21 minutes),
without compacting.
- From the engine log, each attempt was `prompt 109013 tokens ..., 26214 generated in ~250000 ms`.
With the budget honoured, the thinking ends at 8,192 tokens and the summary has room.
The same thing happens to any llama-server client that relies on `thinking_budget_tokens`. The thinking is simply
unbounded, which costs time on every request and fails outright when the client also caps the whole answer.
## The fix
`Service.reasoning_budget()` reads `reasoning_budget_tokens` and, when it is absent, `thinking_budget_tokens`, as
llama-server does. When both are set, `reasoning_budget_tokens` wins. The validation is unchanged: a whole number, and
0 means no budget. The config default still applies when neither field is given.
- `serve/server.py`: 4 lines in `reasoning_budget()`.
- `serve/test_server.py`: `ThinkingBudget.test_llama_cpp_thinking_budget_tokens_is_the_same_budget`. The alias wraps the
thinking up at the same point, and `reasoning_budget_tokens` takes precedence when both are set.
- `docs/DETAILS.md`: the alias is named in the thinking-budget paragraph.
The engine is not touched.
## Testing
`python -m pytest serve/test_server.py`: 297 passed (Windows, Python 3.12). Running on the box above since 2026-10-12.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
No site
Links install, modelos, releases.