#2980 serve: /v1/chat/completions ignores llama.cpp's n_predict, so a client's n_predict: 0 follow-up call regenerates the whole answer with no limit (~20k tokens, ~4–5 min of GPU time per reply)

Beschreibung

## Summary

`/v1/chat/completions` ignores the llama.cpp-native field `n_predict`. The answer limit is read only from `max_completion_tokens` / `max_tokens`, and a missing or 0 value means "the rest of the context" (`serve/server.py:5942` on `main` at `477ac13c`; `:5819` in 0.1.44.1, in `_openai`). Still the case on `main` as of 2026-10-17.

llama.cpp's own web UI (and possibly other llama.cpp-compatible clients) sends a follow-up request after every reply. The request carries the conversation so far, ending with the assistant's answer, plus `n_predict: 0` and `stream: false`. In llama.cpp, `n_predict: 0` means "generate nothing": the client only wants the prompt processed, e.g. to measure context or warm the cache. Strata generates with no limit instead. The model answers the whole task again from scratch, and the client discards the result.

Because Strata serves one request at a time, any other client of the server waits behind this request for those minutes. The chat user can cut it short: sending a new message cancels it (seen twice; it fits the client dropping its pending call, and Strata cancelling a request whose client disconnects). But that has a second cost. The follow-up never completes as a prompt-only call, so the conversation is never cached, and the user's next message re-reads the whole conversation from scratch. Strata already reports `n_predict` in `/props` (`default_generation_settings.params.n_predict`), so llama.cpp clients reasonably expect it to be honoured on requests.

## Environment

- Strata 0.1.44.1 (`296b59d2`, git clone). It first appeared on a `main` ZIP snapshot from 2026-10-04 (bundled engine 0.1.39) and is still there after updating, so it isn't a regression in 0.1.44.x. The same limit line is in both versions (`serve/server.py:3368` in the 2026-10-04 snapshot).
- Windows 10, `backend: hip`, AMD Radeon AI PRO R9700 (gfx1201, 32 GB), 96 GB RAM
- Happens after every reply in that client, across several days and on four packs: `iq3_s`, `coder-iq1_m`, `unsloth-ud-iq4_xs` (Qwen3.8-Flash-Next) and `swift-iq3_s` (Swift-1.5). The cases below are the ones captured in logs. The code path is in the shared request layer, so it does not depend on the model.
- Client: **llama.cpp's built-in web UI ("llama-ui")**. Its chat requests carry `return_progress`, `timings_per_token`, `backend_sampling` and `reasoning_format`; after each reply it sends the `n_predict: 0` call. Strata's own web app never sends `n_predict` and doesn't trigger this.
- How it ended up talking to Strata: llama-ui installs itself as a PWA with a service worker. It was installed from llama-server at `127.0.0.1:8080`, llama-server's default port and also Strata's. When Strata later served that address, `--open` brought up the cached llama-ui shell instead of Strata's page (its icons and bundle 404 against Strata), and llama-ui worked against Strata's llama.cpp-compatible API without any visible sign. Anyone who has used llama-server on 8080 can hit this without choosing that client.

## Minimal repro

Same prompt, non-streaming, against an idle server:

```
POST /v1/chat/completions
{"messages":[{"role":"user","content":"Count from 1 to 40, one number per line. No thinking."}],
 "stream":false, "n_predict": 8}
```

| Limit field | completion_tokens | finish_reason |
|---|---|---|
| `"n_predict": 8` | 155 | `stop` |
| `"max_tokens": 8` | 8 | `length` |
| `"max_tokens": 0` | 153 | `stop` |

## What it looks like in practice (captured with `"api_monitor": true`)

1. The user's chat request: `stream: true`, prompt 1,081 tokens, 29,263 tokens generated.
2. 0.05 s after it finished, the client's follow-up: `stream: false`, `n_predict: 0`. The messages are exactly [user prompt, the previous answer as an `assistant` message]. Prompt 5,763 tokens, 0 reused. Live metrics showed `max_tokens: 518517`. It finished with **21,683 tokens in 292 s**: about 64k characters of reasoning plus a full second answer.

An earlier session the same day showed the same shape: chat prompt 1,081 → 39,593 tokens, then the follow-up at 7,468 prompt tokens, 0 reused, **20,181 tokens in 232 s**. In an earlier occurrence, on the `coder-iq1_m` pack, the user sent a new message about 200 s in, and that ended the follow-up.

### Interrupting it (`swift-iq3_s`, from the server log)

| # | Request | Prompt | Result |
|---|---|---|---|
| 1 | User's chat | 1,081 | 20,201 tokens, `stop` |
| 2 | Follow-up (`n_predict: 0`) | 8,030, 0 reused | cancelled after 4,681 tokens / 46.6 s, when the user sent a new message |
| 3 | User's new message | 21,294, **0 reused** (24.7 s to read) | 30 tokens, `stop` |
| 4 | Follow-up again | 21,014, **0 reused** (23.9 s to read) | 31 tokens, `stop` |

Two things show here:
- **It fires after every reply.** The cost scales with the length of the previous answer: minutes after a long one, 31 tokens after a short one.
- **Nothing is ever reused.** Every resend re-reads the full conversation, because the call that would leave it cached never completes as intended. With `n_predict: 0` honoured, request 2 would have taken about 10 s of prompt reading, and request 3 could have reused most of its prompt.

## History

No version has read `n_predict` from a request. What changed over time is the cost:

- `01529aaa` (2026-09-24, first commit): the limit comes from `max_completion_tokens` / `max_tokens`, with a default of 1024 if neither is set. An `n_predict: 0` call was ignored, but capped at 1,024 tokens.
- `656a8356` (2026-09-26, "treat unset/-1 max_tokens as remaining context, not 1024"): the default became the rest of the context, so the same call now has no limit.
- `08f8d591` (2026-09-29, "add llama.cpp-compatible discovery endpoints"): `/props` started reporting `n_predict`, so llama.cpp clients treat Strata as compatible.

`656a8356` was a sensible change for real chat requests. It just made the missing `n_predict` handling expensive.

## Suggested fix

Honour `n_predict` in `_openai` when neither `max_completion_tokens` nor `max_tokens` is set. Note that this can't reuse the existing `int(... or 0)` pattern. For `max_tokens`, 0 deliberately means "the rest", so passing `n_predict` through it would keep `n_predict: 0` unlimited. llama.cpp's semantics are:

- `n_predict: -1`: no limit
- `n_predict: 0`: process the prompt and generate nothing. Return an empty completion (`completion_tokens: 0`) once the prompt has been read; the client presumably wants the prompt cached.
- `n_predict: N > 0`: same as `max_tokens: N`

Happy to test a fix on gfx1201.

Related: #2141 (same family: `thinking_budget_tokens`, llama-server's name, is ignored, so llama.cpp clients get unbounded thinking). Together, a llama.cpp client that tries to bound a request by either length or thinking gets no bound at all.