#6610 [Engine] sycl serve: take the hy4/inkling keywords the shared serve sends sampling_keys — every chat through server_intel.py was a 400 since v0.2.18

open · @kgmkm · 0 comentários · No GitHub

Setup & installServer & APIAMD / HIPDocumentationWindowsLinux

Descrição

## Problem

Since v0.2.18 (836ab9714, "serve: advertise Hy4-Flash in-process vision and strip unsupported GEN keys"), every chat request through the Intel Arc source serve path (`engine/sycl/serve/server_intel.py`, the wrapper `docs/INTEL.md` documents) answers 400 before the engine sees a line:

```
{"error": {"type": "invalid_request_error",
    "message": "the request is malformed (TypeError: install_logprobs.<locals>.sampling_keys()
    got an unexpected keyword argument 'hy4')"}}
```

The shared serve now calls `StrataEngine.sampling_keys(...)` with `hy4=`/`inkling=` at both call sites under `generate()` — the GEN/GENI head and the conversation-cache key (`keys0`). `install_logprobs()` here shadows the method with a one-argument wrapper, so the call raises and the request dies as a 400. `git diff v0.2.17 v0.2.19 -- engine/sycl` is empty, so the wrapper never saw the new keywords. A mock-based smoke run never enters `generate()`, which is how three releases shipped with this path broken; `/v1/models`, startup and the web page all work, only generation routes fail.

## Result

The wrapper takes the keywords and passes them on. They are `False` on this path unless a config serves strata-hy4 / strata-inkling through this wrapper, and the key strings this returns are unchanged from v0.2.17's wrapper: plain requests build the same GEN head, a logprobs request builds the same key this wrapper always built.

## Approach

One signature, `def sampling_keys(sampling, **kw):` → `keys(sampling, **kw)` + the wrapper's own logprobs suffix. Forwarding, not swallowing: a hy4/inkling config through this wrapper keeps the shared function's key stripping. Plus a regression test that imports this wrapper in a **subprocess** — unittest discovery runs every `test_*.py` in one process, where an earlier test may already hold a different `serve` module object (that is exactly how a naive in-process import of this wrapper fails here); a child interpreter is immune, needs no GPU, and leaves nothing patched behind for the tests that follow. The mock engine's blind spot is also why the test drives the wrapper directly instead of a smoke request.

## Validation

**Passed** (Intel Arc Pro B70 32 GB, `strata-engine-0.1.65-linux-x86_64-sycl`, repo at v0.2.19, Ryzen 7 3700X, Ubuntu 26.04):
- `python -m unittest discover -s terminal/serve -p 'test_intel_serve_sampling_keys.py'` — green with this change; red on the unpatched tree with the production TypeError above.
- Full local suite `discover -s terminal/serve -p 'test_*.py'`: 271 tests, the same 160 environment-dependent import errors before and after (this machine lacks the fixtures those need); the new test passes inside the full run and adds no errors.
- End to end through `server_intel.py --engine strata`: `/v1/chat/completions` greedy and temperature 0.7 both 200 with correct answers; a `logprobs: true` + `top_logprobs: 5` request 200 (400 on the unpatched tree); the Anthropic `/v1/messages` route 200 with a correct answer.

**Failed:** none on the hardware above.

**Not run:** Windows (same wrapper file); strata-hy4 / strata-inkling configs through this wrapper (the keywords are forwarded for them, untested); the packaged Terminal and app paths (they call the shared `sampling_keys` directly and never broke).

## Limits

- A logprobs request is served (200) but its reply carries no logprobs entries on v0.2.19 + engine 0.1.65 — the same with and without this wrapper's suffix, so it is separate from the 400 this fixes and is not touched here.
- The shared `sampling_keys` also appends its own ` logprobs=K` (already at v0.2.17), so this wrapper's suffix doubles it for logprobs requests, exactly as v0.2.17's wrapper did; kept byte-identical on purpose. Dropping the suffix is a separate one-line change if the double key is unwanted.

Searched the open and closed issues and pull requests first: nothing covers this. The nearest is #5086 — the same wrapper family at 0.2.10, where `engine_args` did not take `announce`; that one added the keyword to the wrapper, which is the same shape of fix as here.

No site

Links install, modelos, releases.