Pull requests / #2254
#2254 Add openai_logprobs/: OpenAI-API logprobs + allowed_token_ids for AnyJev, with a bilingual CN/EN launcher
open · @xiezhipeng-git · 0 コメント · GitHub で見る
Setup & installServer & APISecurityDocumentationWindows
本文
## What Adds an `openai_logprobs/` package that gives Strata's `/v1/chat/completions` the OpenAI fields `logprobs`, `top_logprobs` and `allowed_token_ids`, plus a two-step desktop launcher (step 1: 中文 / English; step 2: model source Hugging Face / ModelScope). - **`server_logprobs.py`** — a standalone extension; nothing under `serve/` is edited, so upstream merges stay clean. It translates the OpenAI fields into the engine's `logprobs=K` / `allowed=a,b,c`, and wires the engine's `LP` lines back into OpenAI's `logprobs` response shape. With `allowed_token_ids` it follows vLLM semantics: only the named ids are returned, renormalised over them — which is what an AnyJev backend reads as the probability distribution over candidate options. If the running engine is not the patched `strata-lp` build, the server prints a named diagnostic instead of returning silently empty logprobs (empty looks like "very confident" and would silently skew downstream confidence gating). - **`build_engine.py` + `openai_logprobs/build/`** — emits the patched engine source copy additively; `CMakeLists.txt` gains the `strata-lp` target; `build-strata-lp.bat` / `build-win-env.bat` build it on Windows. `selftest.py` covers the request/response mapping offline. - **`make_lp_config.py`** — derives a model config whose `exe` points at `strata-lp.exe`. - **`start_server.py` + `START-CN.bat`** — the launcher: language choice, then model source (ModelScope is upstream's own feature; the launcher only translates the choice into `setup.py --source`). Falls back to the stock engine when `strata-lp` is not built, so a double-click always starts. The choice is remembered in `openai_logprobs/launch-choice.json`. - **`README.local.md`** documents the setup; **`strata_125b.bat`** is a convenience launcher for a local model pack. ## Why AnyJev-style decision readouts need per-request token log-probabilities, which stock serving does not expose over the OpenAI chat API. This makes a locally served Strata model usable as any AnyJev backend: point it at `http://127.0.0.1:8080/v1` and read option probabilities straight from the response. ## Verification - `openai_logprobs/selftest.py`: 18/18 on Windows. - The `strata-lp` engine builds on Windows (`build-strata-lp.bat`), and the logprobs response shape and its renormalisation were verified end-to-end against that engine (documented in `README.local.md`). - The client-side contract is cross-checked field-by-field against AnyJev's backend consumer (`allowed_token_ids` in, `choices[0].logprobs.content[*].top_logprobs` with authoritative token ids out).
関連リンク
インストール・モデル・リリースへの站内リンク。