Pull requests / #3340
#3340 [Other] serve: accept pi's three thinking-budget field names as aliases
open · @shizhx · 0 comments · View on GitHub
Server & APIMulti-GPUAMD / HIPNVIDIA / CUDAModels & quantsDocumentationLinux
Description
Problem and result: pi's `compat.thinkingTokenBudgetField` only allows
`thinking_token_budget` (vLLM), `thinking_budget` (Qwen/DashScope/SGLang) and
`thinking_budget_tokens` (llama.cpp), but #123's thinking budget was only readable
from `reasoning_budget_tokens`, so a pi client's budget was silently ignored - the
model thought its full length instead of being wrapped up at N tokens. After this
change all three names work exactly like `reasoning_budget_tokens` on the three
chat routes. Related: issue #123 #984.
## Summary
```diff
_post (serve/server.py)
req = json.loads(body)
if path in chat routes:
self.record = svc.begin_request(path, req)
+ if req.get("reasoning_budget_tokens") is None:
+ first of thinking_token_budget / thinking_budget / thinking_budget_tokens
+ -> req["reasoning_budget_tokens"]
dispatch -> _responses / _openai / _anthropic
svc.reasoning_budget(req) # existing: reads the native name, 400 on a non-integer
```
One commit, 3 files, +49/-1 (`serve/server.py` alias copy, `serve/test_server.py`
4 tests, `docs/DETAILS.md` the budget bullet). A request that sends none of the
three names goes through the added lookups unmodified: no mutation, no config,
engine or response change, so the default path is unchanged.
## Evidence
- **Before:** on base e88ee39d the three names are silently ignored
```text
# fake-engine suite: the new tests against the unpatched server
FAIL: test_the_budget_alias_field_names
AssertionError: 'Let me think step by step about two plus two. ...' !=
'Let me think step by\n\nI have thought about this long enough ...'
FAIL: test_a_bad_alias_is_a_400
AssertionError: 200 != 400 : ('thinking_token_budget', 1.5, ...)
Ran 23 tests in 13.215s - FAILED (failures=2)
```
- **After:** e88ee39d + this commit
```text
test_the_budget_alias_field_names red -> green
test_a_bad_alias_is_a_400 red -> green
test_the_request_s_own_budget_wins_over_the_aliases green (precedence guard)
test_the_alias_field_order_wins green (precedence guard)
Ran 23 tests in 13.231s - OK # serve.test_server.ThinkingBudget
Ran 885 tests in 377.483s - OK (skipped=14) # full serve suite, CI's command
```
live on :4000 (real IQ3_S model, 2x RTX 2080 Ti), prompt "What is 17*23? ...",
budget 20:
```text
no budget (baseline) 200 finish=stop reasoning_chars=183, no wrap-up
native 200 finish=stop wrapped=True reasoning_chars=181 answer='17 × 23 = 391'
thinking_token_budget 200 finish=stop wrapped=True reasoning_chars=134 answer='17 × 23\n...'
thinking_budget 200 finish=stop wrapped=True reasoning_chars=178 answer='17 × 23:...'
thinking_budget_tokens200 finish=stop wrapped=True reasoning_chars=134 answer='...'
bad alias (1.5) 400 reasoning_budget_tokens=1.5: expected a whole number of tokens
```
Each alias wraps the thinking at 20 (wrap-up marker, answer after, finish stop)
exactly like the native field; the response shape is unchanged.
## Approach
The alias copy happens once, at the request entry (`_post`, after `begin_request`,
before dispatch), so every downstream read - `Service.reasoning_budget` on all
three routes, including `/v1/responses` - sees the native name and keeps the
existing whole-number validation (a bad alias is the same 400). Precedence is
deterministic: the request's own `reasoning_budget_tokens` wins over all aliases;
among the aliases the fixed order `thinking_token_budget` > `thinking_budget` >
`thinking_budget_tokens` does. The patch applies verbatim onto e88ee39d: `do_POST`'s
body moved to `_post` and the req parse now happens after `reserve_inference`, but
the insertion point and the budget's read path are unchanged.
## Validation
Revision e88ee39d + 2b336465, Python 3.13.5,
Linux x86-64, 2x RTX 2080 Ti, Qwen3.8-Flash-Next IQ3_S:
- Passed:
- `.venv/bin/python -m unittest serve.test_server.ThinkingBudget -v` - red
before (the alias-wrap and bad-alias-400 tests fail on base; the two
precedence tests pass since the native field already wins when the aliases
are ignored) / green after, 23 OK
- `.venv/bin/python -m unittest discover -s serve -p 'test_*.py'` (CI's
command): 885 OK (14 skipped)
- live requests on :4000 (`strata serve --profile strata-iq3_s.json`, the
patched tree): see Evidence
- Failed: none observed
- Not run: no C++ backend is touched (serve/ Python only), so no CUDA/HIP/SYCL
builds; the live check used the OpenAI route - the Anthropic route and
`/v1/responses` are covered by the suite's fake-engine tests only
## Merge Danger
**Door:** two-way - revert the one commit; no data, schema, config or feed
migration.
**Blast Radius:** request handling on the three chat routes. A client that sends
none of the three names is unaffected (lookups only, no mutation); the only
behavior change is that a previously-ignored field now does what its name says.
## Limits
- Several aliases at once: the first of the fixed order does (documented in
DETAILS.md).
- The aliases apply on all three chat routes, so an Anthropic body carrying a
top-level `thinking_budget` is aliased too, same as the native field's
behavior there.
- The live check covered one model (IQ3_S) and one prompt family; the routes and
response shapes are covered by the suite.Related on this site
Quick links to related pages.