#6402 [Engine] serve: /v1/models reports the context window but no output-token limit - clients guess the answer cap
open · @nikola-andrijevic · 0 评论 · 去 GitHub 看
Setup & installServer & APIAMD / HIPModels & quantsWindows
说明
### Area
[Engine] - the server and its API (`terminal/serve/server.py`, `/v1/models`).
### Desktop app, Terminal or source?
Terminal (the `strata` command, serving through `terminal/serve/server.py`).
### What happened
`GET /v1/models` says how big the window is, and says nothing about how long one answer may be. In v0.2.17 the fields published per model are (server.py, ~line 6984):
```python
"max_model_len": ctx, "context_length": ctx, "max_context_length": ctx, "meta": {"n_ctx": ctx}
```
The context half of #5945 is now fixed (#6116 shipped these names in 0.2.17). The output half has no field at all - no `max_output_tokens`, no `limit.output`, nothing equivalent. The server enforces the config's `max_tokens` (32,768 here), so the answer budget is real and known to the server - it is just never published.
A discovering client then guesses. Measured with the OpenCode models-discovery plugin (installed dist, v2.2.0): its llama-swap reader takes `meta.llamaswap.max_output_tokens` only; when absent it falls back to `DEFAULT_OUTPUT_TOKEN_LIMIT = 32e3`. Result on this PC: the client reports **32,000** answer tokens instead of the real **32,768** - and for any config whose `max_tokens` is lower than 32,000 the guess would be too high, so the client over-promises and answers get cut mid-sentence. Before the field existed, older clients fell back to 1,024 and cut every answer at 1,024 tokens: same class of bug #5945 describes for context.
Expected: `/v1/models` reports the per-model output cap next to the context window. Suggested shape (the names clients already look for):
```python
model["limit"] = {"output": svc.output_cap} # OpenAI-style / OpenCode
model["meta"]["llamaswap"] = {"max_output_tokens": svc.output_cap} # llama-swap
```
The value is already in the config (`max_tokens`); `fit_max_tokens` already trims each request to the remaining room.
### Smallest steps or command that reproduce it
On any v0.2.17 checkout, no GPU needed:
```powershell
git show v0.2.17:terminal/serve/server.py | Select-String -Pattern 'max_output' # nothing in /v1/models
```
With one serving model:
```
GET /v1/models -> max_model_len / context_length / max_context_length / meta.n_ctx published;
no output field anywhere in the model object
```
### Reproduction status
Reproduced locally
### Strata version
0.2.17 (engine 0.1.63), Windows. The clean tree was read with `git show v0.2.17:terminal/serve/server.py`.
### Graphics card, memory and system
AMD Radeon RX 9070 XT (gfx1201) - Ryzen 5 9600X - 32 GiB RAM - Windows 11 26H2 (build 26200)
### Model, size and settings
Qwen3.8-Flash-Next GSQRCO IQ1_M - context 262,144 - config `max_tokens` 32,768 - API key configured
### Logs
Live `GET /v1/models` on this PC. My copy carries a local two-line patch that publishes the field; that is why it appears here:
```json
{"id": "qwen3.8-flash-next-coder-iq1_m",
"max_model_len": 262144, "context_length": 262144, "max_context_length": 262144,
"meta": {"n_ctx": 262144, "llamaswap": {"max_output_tokens": 32768}},
"limit": {"output": 32768}}
```
On the untouched v0.2.17 tree the same call omits `limit` and `meta.llamaswap` entirely.
Limits of what I measured: Windows, one model and config; the plugin fallback values were read from its installed dist (v2.2.0), not from issue reports. Related but different: #5945 and #6116 (context half), #4230 and #5006 (config-side output caps in `strata connect`), #5554 (desktop sends `max_completion_tokens: 1`).本站相关内容
相关页面的快捷入口。