#5836 [Engine] Native /v1/models still has no context window on 0.2.13 — engine reports n_ctx, the host gateway drops it
open · @realbutnice · 0 评论 · 去 GitHub 看
Server & APINVIDIA / CUDAModels & quantsSecurityWindows
说明
**Area:** `[Engine] (speed, kernels, models, the server and API)`
**Desktop app, Terminal or source?:** Desktop app (Windows)
---
**What happened**
I loaded `Qwen3.8-Flash-Next IQ3_XXS` (131072 context, images on) in the desktop app, then asked the host gateway for the model list:
```
GET http://127.0.0.1:47831/v1/models
{"object":"list","data":[{"id":"iq3_xxs","object":"model","owned_by":"strata",
"meta":{"instanceId":"…","gpus":[0],"label":"Qwen3.8-Flash-Next IQ3_XXS","aliases":["iq3_xxs"]}}]}
```
There is no `n_ctx`, no `context_length`, no `max_model_len` — nothing a client can size a context window from.
The engine behind it does report the value, three ways:
```
GET http://127.0.0.1:18500/v1/models → "meta": {"n_ctx": 131072}
GET http://127.0.0.1:18500/props → "default_generation_settings": {"n_ctx": 131072}
GET http://127.0.0.1:18500/metrics → "engine": {"max_context": 131072}
```
So the data exists at the engine and is dropped when the host builds its own model list. What I expected instead: the host's `/v1/models` entries carry the loaded instance's context window under `meta.n_ctx` plus `context_length` and `max_model_len` (the names #1882 proposes), on the primary entry and on alias entries.
Why it matters: OpenAI-compatible agent clients size their context window from this list. When the key is missing they fall back to their own default — in my case 1M against a 128K engine — so they never compact and the conversation is hard-truncated at 131072 instead of being summarized before the limit. Working around it means hand-configuring the context window in every client, on every machine, with no way to keep them consistent.
`GET http://<lan-ip>:47832/v1/models` (the share listener) returns the same `meta` with exactly those four keys, so sharing hits the same gap.
This is the same report as #3931, which was closed by the stale-issue sweep rather than by a fix, so I re-tested on 0.2.13 as that comment asked. Still reproduces.
---
**Smallest steps or command that reproduce it**
1. Load IQ3_XXS in the app (Settings > Models > Load).
2. `curl -s -H "Authorization: Bearer $KEY" http://127.0.0.1:47831/v1/models` → the response above, with no context key.
3. For comparison, read the engine directly: `curl -s -H "Authorization: Bearer $ENGINE_KEY" http://127.0.0.1:18500/v1/models` → `meta.n_ctx: 131072`.
(`$KEY` is the serving key from `/api/serving`; `$ENGINE_KEY` is `api_key` in `native-instances/<instanceId>.json`.)
**Reproduction status:** Reproduced locally
**Strata version:** Strata 0.2.13 · engine 0.1.59 (windows-x86_64-cuda)
**Graphics card, memory and system:** RTX 5060 Ti 16 GB · 96 GB RAM · Windows 11 Pro
**Model, size and settings:** Qwen3.8-Flash-Next IQ3_XXS · 131072 context · pictures on
**Logs:** can attach `strata-iq3_xxs.log` if useful — nothing in it looks related, the engine reports the value fine.
---
**Where the fix would live (for the maintainers)**
`host/server/index.mjs`, the native branch of `/v1/models` (line ~2145 in 0.2.13). The instance already holds the value and the same file computes it elsewhere for `localInstances` as `i.metrics?.engine?.max_context ?? i.context`; `host/server/model-providers.mjs:143` already reads `context_length` from remote providers, so the naming convention is in the project.
Current:
```js
.map(i=>({id:i.apiModelId,object:'model',owned_by:'strata',meta:{instanceId:i.id,gpus:i.gpus,label:i.label,aliases:i.aliases}}))
```
Proposed:
```js
.map(i=>{const ctx=i.metrics?.engine?.max_context??i.context??0;return{id:i.apiModelId,object:'model',owned_by:'strata',
context_length:ctx||undefined,max_model_len:ctx||undefined,max_input_tokens:ctx||undefined,
meta:{n_ctx:ctx||undefined,instanceId:i.id,gpus:i.gpus,label:i.label,aliases:i.aliases}}})
```
Keys are omitted rather than sent as 0 when the value is not known yet (metrics are polled every 2s and can be briefly null right after load). I can send this as a PR if you prefer a patch over an issue.
本站相关内容
相关页面的快捷入口。