#6402 [Engine] serve: /v1/models reports the context window but no output-token limit - clients guess the answer cap

open · @nikola-andrijevic · 0 comments · View on GitHub

Setup & installServer & APIAMD / HIPModels & quantsWindows

Description

### Area
[Engine] - the server and its API (`terminal/serve/server.py`, `/v1/models`).

### Desktop app, Terminal or source?
Terminal (the `strata` command, serving through `terminal/serve/server.py`).

### What happened
`GET /v1/models` says how big the window is, and says nothing about how long one answer may be. In v0.2.17 the fields published per model are (server.py, ~line 6984):

```python
"max_model_len": ctx, "context_length": ctx, "max_context_length": ctx, "meta": {"n_ctx": ctx}
```

The context half of #5945 is now fixed (#6116 shipped these names in 0.2.17). The output half has no field at all - no `max_output_tokens`, no `limit.output`, nothing equivalent. The server enforces the config's `max_tokens` (32,768 here), so the answer budget is real and known to the server - it is just never published.

A discovering client then guesses. Measured with the OpenCode models-discovery plugin (installed dist, v2.2.0): its llama-swap reader takes `meta.llamaswap.max_output_tokens` only; when absent it falls back to `DEFAULT_OUTPUT_TOKEN_LIMIT = 32e3`. Result on this PC: the client reports **32,000** answer tokens instead of the real **32,768** - and for any config whose `max_tokens` is lower than 32,000 the guess would be too high, so the client over-promises and answers get cut mid-sentence. Before the field existed, older clients fell back to 1,024 and cut every answer at 1,024 tokens: same class of bug #5945 describes for context.

Expected: `/v1/models` reports the per-model output cap next to the context window. Suggested shape (the names clients already look for):

```python
model["limit"] = {"output": svc.output_cap}                         # OpenAI-style / OpenCode
model["meta"]["llamaswap"] = {"max_output_tokens": svc.output_cap}  # llama-swap
```

The value is already in the config (`max_tokens`); `fit_max_tokens` already trims each request to the remaining room.

### Smallest steps or command that reproduce it
On any v0.2.17 checkout, no GPU needed:

```powershell
git show v0.2.17:terminal/serve/server.py | Select-String -Pattern 'max_output'   # nothing in /v1/models
```

With one serving model:

```
GET /v1/models -> max_model_len / context_length / max_context_length / meta.n_ctx published;
                  no output field anywhere in the model object
```

### Reproduction status
Reproduced locally

### Strata version
0.2.17 (engine 0.1.63), Windows. The clean tree was read with `git show v0.2.17:terminal/serve/server.py`.

### Graphics card, memory and system
AMD Radeon RX 9070 XT (gfx1201) - Ryzen 5 9600X - 32 GiB RAM - Windows 11 26H2 (build 26200)

### Model, size and settings
Qwen3.8-Flash-Next GSQRCO IQ1_M - context 262,144 - config `max_tokens` 32,768 - API key configured

### Logs
Live `GET /v1/models` on this PC. My copy carries a local two-line patch that publishes the field; that is why it appears here:

```json
{"id": "qwen3.8-flash-next-coder-iq1_m",
 "max_model_len": 262144, "context_length": 262144, "max_context_length": 262144,
 "meta": {"n_ctx": 262144, "llamaswap": {"max_output_tokens": 32768}},
 "limit": {"output": 32768}}
```

On the untouched v0.2.17 tree the same call omits `limit` and `meta.llamaswap` entirely.

Limits of what I measured: Windows, one model and config; the plugin fallback values were read from its installed dist (v2.2.0), not from issue reports. Related but different: #5945 and #6116 (context half), #4230 and #5006 (config-side output caps in `strata connect`), #5554 (desktop sends `max_completion_tokens: 1`).

Related on this site

Quick links to related pages.