#6180 [Other] Control-app /v1/models strips meta.n_ctx/status/architecture the engine exposes — breaks context-window auto-detection for OpenAI-compatible clients

open · @yoshimitsuspeed · 3 Kommentare · Auf GitHub

Setup & installServer & APINVIDIA / CUDAModels & quantsSecurityLinux

Beschreibung

Connecting Hermes to Unsloth Desktop models I could leave context window in Hermes at 0 (auto detect) and it would pick up the context window size of the model loaded in UD. 

This does not work in Strata when I load the model through the Strata UI. 

Here is my local agent's report on the issue. 

Native-serving control app `/v1/models` omits `meta.n_ctx`, `status`, `architecture`, and alias entries that the engine exposes — breaking context-window auto-detection for OpenAI-compatible clients

## Environment
- Strata app 0.2.13, control 0.1.59.1, engine 0.1.60-linux-x86_64-cuda-cuda13, Linux
- **Not fixed in latest:** verified against 0.2.15 release code (main @ c775728069, 2026-11-16) —
  the native `/v1/models` handler is byte-identical (index.mjs:2108), `n_ctx` appears nowhere in
  `apps/host/server/*.mjs`, and `/health` is not proxied. 0.2.14/0.2.15 release notes contain no
  API-surface changes (prompt kernels, drafter persistence, installers).
- Native instance: `swift-iq3_s` pack, `--max-context 262144`, GPU 1, vision enabled
- Control app serving on `127.0.0.1:47831`; engine instance on `127.0.0.1:18500`

## Reproduction
```bash
# Engine (instance port) — full metadata:
curl -s -H "Authorization: Bearer <engine key>" http://127.0.0.1:18500/v1/models
# Control app (serving port) — metadata stripped:
curl -s -H "Authorization: Bearer <serving key>" http://127.0.0.1:47831/v1/models
curl -s -H "Authorization: Bearer <serving key>" http://127.0.0.1:47831/health   # 404/HTML
```

### Engine :18500 returns (3 entries: canonical + 2 aliases)
```json
{"id":"swift-1.5-iq3_s","object":"model","status":{"value":"loaded"},
 "meta":{"n_ctx":262144},
 "architecture":{"input_modalities":["text","image"],"output_modalities":["text"]},
 "aliases":["Strata","swift-iq3_s"]}
```
plus full entries for `Strata` and `swift-iq3_s` with `"alias_of":"swift-1.5-iq3_s"`.
`GET :18500/health` → `{"status":"ok","max_context":262144,"model":"swift-1.5-iq3_s","images":true,"loaded":true,...}`
`GET :18500/props` → `default_generation_settings.n_ctx: 262144`, chat template, slots.

### Control app :47831 returns (1 entry, all capability metadata gone)
```json
{"id":"swift-iq3_s","object":"model","owned_by":"strata",
 "meta":{"instanceId":"e02a266a-...","gpus":[1],"label":"Swift 1.5 IQ3_S",
         "aliases":["Strata","swift-iq3_s"]}}
```

## What is stripped (engine → control app)
| field | engine :18500 | control :47831 | who needs it |
|---|---|---|---|
| `meta.n_ctx` (context window) | 262144 | **absent** | every OpenAI-compatible client that auto-detects context (Hermes Agent, Open WebUI, aider, …) |
| `status.value: loaded` | present | **absent** | "is the model ready" checks, routers, guardians |
| `architecture.input_modalities` | `["text","image"]` | **absent** | vision-capability detection |
| alias entries (`Strata`, `swift-iq3_s` with `alias_of`) | full entries | only canonical id + `meta.aliases` | clients that match by id, picker dedup |
| canonical id | `swift-1.5-iq3_s` | reported as `swift-iq3_s` (apiModelId) | id stability across surfaces |
| `/health` | JSON w/ `max_context` | 404/HTML | health probes (Hermes fleet scripts use it) |
| `/props` | llama.cpp-compatible | absent | llama.cpp-style auto-detect probes |

## Root cause (source, public repo)
`apps/host/server/index.mjs:2108` (native branch of the OpenAI-compatible router):
```js
if(path==='/v1/models'&&method==='GET')return json(res,200,{object:'list',
  data:native.state().instances.filter(i=>i.status==='ready').map(i=>(
    {id:i.apiModelId,object:'model',owned_by:'strata',
     meta:{instanceId:i.id,gpus:i.gpus,label:i.label,aliases:i.aliases}}))});
```
The control app **synthesizes** `/v1/models` from its own instance registry instead of
passing through (or merging) the engine's response. Nothing in that map carries context,
status, modality, or alias entries.

Notably, the app already HAS the data: the runtime-mode handler two blocks below
(index.mjs:2115) emits `meta.context`, and index.mjs:217/638 use
`i.metrics?.engine?.max_context` for UI display and frame metadata.

## Impact (concrete, measured)
Clients that auto-detect context from `/v1/models` (the de-facto OpenAI-compat mechanism)
get no number and fall back to catalogs/guesses. Observed with Hermes Agent:
- model `deepseek-v4.1-flash-ist-das-2.25bpw` → client catalog match → assumed **1,000,000**
  token window while Strata serves 262,144 → context compaction never triggers →
  hard context overflow mid-session, runaway summarizer calls, session crash.
- model `swift-iq3_s` → falls back to a 256,000 guess (close, but fiction).
The engine already reports the truth at `meta.n_ctx`; one stripped field cost us multiple
lost work sessions.

## Proposed fix (small, one-line class of change)
1. In the native `/v1/models` handler, include per-instance truth:
   `meta.n_ctx` (from `i.metrics.engine.max_context`, already tracked),
   `status:{value:"loaded"}`, `architecture.input_modalities` (from the vision flag),
   and emit alias entries with `alias_of` — mirroring what the engine returns.
2. Proxy `GET /health` (and optionally `/props`) to the ready native instance.
3. (Optional) pass through unknown `meta` keys from the engine's `/v1/models` so future
   engine fields reach clients without app changes.

## Asks
- Accept `body.model` equal to any listed alias on the chat path (canonicalize via `alias_of`).
- Document that the control-app `/v1/models` is a registry summary, not the engine list,
  so integrators don't probe it expecting engine metadata.

## Notes for triage
- Engine binary needs no changes; fix is JS-only in `apps/host/server`.
- The runtime-mode handler (index.mjs:2115, `meta.context`) shows the intended shape.
- Happy to test a build; we run 0.2.13 native on Linux with 3090/3060 lanes.

Mehr auf der Site

Links zu Install, Modellen, Releases.