改动 / #2909

#2909 serve: an opt-in strict_model that refuses a request naming another model

open · @BoogeAI · 0 评论 · 去 GitHub 看

Setup & installServer & APIAMD / HIPNVIDIA / CUDAWindows

说明

## Title

serve: an opt-in `strict_model` that refuses a request naming another model

Issue: Refs #2717

## Summary

The three client routes answer **any** model name with whatever is loaded, and the reply's own `model` is the
server's, so a client pointed at another local model (a fine-tune vs the base, two configs on one port) cannot
tell it is being answered by the wrong weights. `/props` has always refused an unknown name; this makes that
opt-in for the routes a client actually talks to, so a mismatch fails loudly instead of silently.

`"strict_model": true` in `strata-<model>.json`: a request whose `"model"` is neither the served name nor an
alias is refused with the 404 `/props` already uses. Absent or `false` keeps today's path exactly as it is; a
request with no `model`, an empty one or a non-string is still accepted, because every route has always read
that as "this server's model" (`Service.model_for`).

## What changed

- **`serve/server.py`** (+27)
  - `Service.strict_model` (default `False`) and `Service.model_is_served(req)`: `False` only when the request
    names a non-empty string that is not in `model_names()`.
  - The check sits where the three routes are dispatched, next to `begin_request`: `404` with
    `{"error": {"type": "invalid_request_error", "code": "model_not_found", "message": "model not found"}}`
    (`responses_error_body(...)` on `/v1/responses`, as that route already shapes its errors), and one line in
    the log naming the asked and the served models.
  - `main()` reads `strict_model` (a non-boolean stops the start, like `reasoning_close_retry`) and prints one
    line when it is on.
- **`serve/test_server.py`** (+60): a `StrictModel` case over all three routes - the default still answers any
  name, the opt-in refuses another name, and the served name / no model / an empty model are accepted.

## Extra Notes

**How I checked.** Measured against a real server (0.1.45 prebuilt CUDA 13.0, RTX 5080 16 GB, Windows 11
26200, a Qwen3.8-Flash-Next fine-tune loaded):

| request `model` | `strict_model` off (before) | on |
| --- | --- | --- |
| the served name | 200 | 200 |
| another model's name (`/v1/chat/completions`, `/v1/messages`, `/v1/responses`) | 200, answered by the loaded model | **404** on all three |
| a deliberate nonsense name | 200 | **404** |
| no `model` field / `"model": ""` | 200 | 200 |

The 404 body on the chat and Anthropic routes is the one above; on `/v1/responses` it is
`{"error": {"message": "model not found", "type": "invalid_request_error", "param": null, "code": "model_not_found"}}`.

**What it leaves alone.** The engine, the kernels, setup, `/props` and `/v1/load|unload` (which already
checked), and the behaviour of every request when the config does not set the key - no name is looked at then.
`/v1/messages/count_tokens` is deliberately not gated: it does not answer with weights.

**In the field.** A front-end that declares both local models on one route and a picker that says "the
fine-tune" while the base model is loaded: with the key on, that pick now returns 404 instead of an answer from
the wrong weights (this is what prompted the issue).

**Not tested.** AMD HIP and SYCL (CUDA only here); an aliased model (the configs I have carry no `aliases`, so
`model_is_served` only ever sees the name) - the alias case is covered by using `model_names()` unchanged.

本站相关内容

相关页面的快捷入口。