Issues / #2717
#2717 [Feature]: an opt-in setting to refuse a request whose model is not the loaded model
open · @BoogeAI · 0 comments · View on GitHub
Setup & installServer & APIAMD / HIPNVIDIA / CUDAModels & quantsDocumentationWindows
Description
### What I want to do
When a client asks this server for a model name it does not serve, the chat routes answer anyway with whatever model is loaded, and the reply reports the server's own name. I would like an opt-in setting that refuses such a request instead — the same `model not found` `/props` already gives — so a client pointed at the wrong local model fails loudly instead of being answered by the wrong weights.
### The case that made me look
I run two local models through one port: the base **Qwen3.8-Flash-Next GSQ-RCO IQ3_S** (`strata-iq3_s.json`) and a community fine-tune (**RVN abliterated**, `strata-rvn-iq3_s.json`). They are mutually exclusive on `127.0.0.1:8080` — I stop one and start the other (~47 GiB against ~53 GiB of experts in RAM). My agent front-end has both declared as two models on one OpenAI-compatible route.
The picker said "RVN", the answers came, and I assumed I was talking to the fine-tune. I was not: the base model was loaded and it answered everything. The fine-tune's style and its (abliterated) refusals never appeared, and nothing in the response said so — `model` in the reply is always the server's own. I only found out by reading `/health` next to the picker.
The documented "any model name" behaviour is good for a one-model install and for clients that hardcode a name (`docs/AI_SETUP.md` §8: "any model name"), so I do not want the default changed.
### What the server does today
Measured on 0.1.44.1 (prebuilt CUDA 13.0 engine), RTX 5080 16 GB, Windows 11 26200, base model loaded, `model` set to a deliberately nonsense name (`完全不存在-的模型名`) and 8 output tokens:
| route | result |
| --- | --- |
| `POST /v1/chat/completions` | 200, `model` in the reply = `qwen3.8-flash-next-iq3_s` |
| `POST /v1/messages` (Anthropic) | 200, `model` in the reply = the same |
| `POST /v1/responses` | 200, `model` in the reply = the same |
| `GET /props?model=<anything else>` | 404 `{"error": {"message": "model not found"}}` — this one does check (`serve/server.py` `_props`, the `model not in svc.model_names()` branch) |
`POST /v1/load` / `/v1/unload` check too, but only for the local page. The three routes a client actually talks to do not.
### What I would like
An opt-in config key, for example
```json
"strict_model": true
```
so that a request whose `model` is neither the served name nor its alias is refused with the 404 body `/props` already uses, on the three client routes. Absent or false keeps the current path exactly as it is.
A list spelling would be fine too (`"accept_model_names": ["strata", "<the model name>"]`) if an alias should keep working; I have no preference between them.
### Workaround I use today
- Split the front-end's provider into one route per local model, with a display name that says which server must be running, and only that model's id in each route's list.
- Read `/health`'s `model` field (my own launcher prints it as a status line) before trusting a picker.
### Scope and what I did not test
Server only (`serve/server.py`); no engine, kernel or setup change. If a mismatched name is refused before the request reaches the engine, nothing an accepted file does can move.
Not tested: AMD HIP and SYCL (this PC is CUDA only); the Anthropic and Responses routes beyond the one mismatch request each above (the three routes share the request path, so one check would sit there); and `POST /v1/embeddings` — it answered 404 to the fake name, but I did not send the served name to it, so that 404 is not evidence either way.
Related on this site
Quick links to related pages.