#6116 [Engine] serve: expose context_length/max_context_length on /v1/models entries (#5945)
closed · @FluxNimbusAi · 0 Kommentare · Auf GitHub
Setup & installServer & APIModels & quantsWindows
Beschreibung
Fixes #5945
Cause: `/v1/models` currently exposes the active context window only under `meta.n_ctx`. Editor clients (`lmstudio-vscode-chat`, `Continue` < v2.26.0) inspect top-level `context_length` / `max_context_length` on each model entry at list time and fall back to a hardcoded `32768` when those keys are absent.
Fix: Mirror `svc.reported_ctx()` into top-level `context_length` and `max_context_length` fields on each `/v1/models` entry in `terminal/serve/server.py` alongside `meta.n_ctx` (inherited automatically by alias entries).
Verification:
- Patched `serve` run on Windows 11 (mock engine): `GET /v1/models` emits `"context_length": 32768, "max_context_length": 32768` alongside `"meta": {"n_ctx": 32768}` on the primary entry — all three read the same `reported_ctx()` value, so any live window is mirrored verbatim.
- Alias entries inherit both keys structurally: the listing composes them with `{**model, "id": x, ...}` (terminal/serve/server.py:6934).
- Pre-fix shape reproduced against a live 65536-ctx serve (release 0.1.55): entry keys are `[architecture, id, meta, object, status]` — no `context_length`. That is the exact fallback trigger; with this change the identical `reported_ctx()` value surfaces under both top-level names.
- The head was rebased onto the 0.2.16 layout (serve/ moved to terminal/serve/); the diff is unchanged at one added line, terminal/serve/server.py:6924.
Limits: one file, one added line, purely additive — no other endpoint changes. Not run: consumer-extension GUI E2E on a workstation with the VSIX installed (the fix targets the field names that consumer source parses at listing time); the 65536 observation comes from a pre-patch live serve rather than a patched-65536 relaunch.
Mehr auf der Site
Links zu Install, Modellen, Releases.