#2170 serve: fix a race that makes a long prompt read give way to a short request one chunk late, or never; its TTFT -64% where it hit (#656)

Description

## Title
Issue: Related: #656 (a long prompt read gives way to a much shorter request; closed when the idea shipped as `BYIELD`
in 0.1.39, the code this fixes) and #2050, where @blange48 found that with `STRATA_PIPED_READ_PIECE=8192` a 42.7K
read gave way only at 16,384 tokens. That was this race in the server, and it is on main too: on one GPU, a short
request misses the boundary it should get in most cases. Overlaps #1288 (which waiting requests count as shorter) and
#2109 (request priority): they change *what* the two give-way checks test, this changes *when* they run. Both already
need a rebase on main; after this one, their new condition goes into `give_way()` / `ask()`. Searched 2026-10-12 for
`BYIELD`, "gives way", `_shorter_waiting` and "heartbeat PP".

## Summary
**In short:** the server asks a long read to give way only when the engine prints a progress line (`PP`). On a
`--batch` server with one GPU, every chunk of a read is a boundary whose only `PP` line comes at its end. The `BYIELD`
answers that line, and it reaches the engine after the engine has looked. So a short request waits one chunk more, or
for the whole read when that was the last boundary the read could give way at. The server now also asks on the
half-second poll it already runs while the engine is quiet. On one RTX 5090 this cut late give-ways from 28 of 40 runs
to 4. **Where the race hit, the short request's TTFT fell 64%: from 4.51 s to 1.64 s, the median of 24 paired runs.**
Server only, +70/-17 lines with tests.

**Why it matters:** the engine looks for a `BYIELD` at each boundary, right after the chunk's `PP` line:

```mermaid
sequenceDiagram
    participant C as short request
    participant S as server
    participant E as engine
    Note over E: reads chunk k (its only PP line comes at its end)
    C->>S: arrives, waits (its prompt is under half as long)
    alt main
        E->>S: PP (chunk k done)
        E->>E: boundary k: no BYIELD yet, reads chunk k+1
        S->>E: BYIELD (the answer to that PP)
        Note over E: gives way at boundary k+1, or never if k was the last it could
    else with this change
        S->>E: BYIELD (quiet poll, within 0.5 s)
        E->>S: PP (chunk k done)
        E->>E: boundary k: gives way (YIELDED)
    end
```

On main the `BYIELD` was in time only at 16,384. There the engine saves a periodic checkpoint after printing the
`PP` line (`--prompt-cache-every`, default 16,384), which keeps it busy until the `BYIELD` is in. The same race hits
#2050's pipelined pieces of one chunk.

**How:** `_control()` already waits for the engine's lines with a 0.5 s timeout. On a timeout it now calls
`on_quiet()`, and the solo read and the admission read pass the same check they run at a `PP` line. Nothing else
changes: the rule for who counts as shorter, `YIELDS_MAX`, one `BYIELD` per read, and what happens when the read ends
before the engine sees it. The engine drops it (`generate.cpp` main loop), and a solo read's reserved slot is freed at
its `DONE`, as before.

**Measured on one RTX 5090** (Swift 1.5 IQ3_S, `--batch 4`, resident experts, 8K chunks of about 1.9 s; the engine
built from main `a0116b27`), main's server and this one. Each server got two engine starts, in the order main, this,
this, main. A one-line request was sent d s into a cold ~33K read, at 12 delays, twice each. The runs were done with
nothing else decoding (the read is a solo `GEN`, `give_way()`) and with one stream decoding beside it (an admission,
`ask()`). Each run's times came from a log of the server's control events, described below.

| | main | with this change |
|---|---:|---:|
| late: a boundary passed after the short request started waiting | 28 of 40 | **4 of 40** |
| of those, never gave way (it waited for the whole read) | 12 | **0** |
| `BYIELD` sent after the request started waiting, median (max) | 1.04-1.58 s (2.71) | **0.22 s (0.46)** |
| short request's TTFT where main was late (nothing decoding / one stream) | 3.56 / 4.40 s | **1.70 / 1.57 s** |
| short request's TTFT where main never gave way | 4.60 / 7.59 s | **1.51 / 2.35 s** |
| both requests' TTFT where both gave way at the same boundary | | the same (medians within 0.02 s) |

The 4 late runs with this change were requests that started waiting 0.02-0.10 s before that boundary's `PP` line,
too close for any poll. Every request that came 0.34 s or more before a boundary got it.

<details><summary>Per delay: where the read gave way and the short request's TTFT (mean of the 2 runs)</summary>

Nothing else decoding:

| sent after | main: gave way at | main: its TTFT | with: gave way at | with: its TTFT |
|---|---|---:|---|---:|
| 0.20 s | 16,384 (late) | 4.36 s | **8,192** | **2.41 / 7.66 s** |
| 0.75 s | 16,384 (late) | 3.56 s | **8,192** | **1.70 s** |
| 1.30 s | 16,384 (late) | 3.05 s | **8,192** | **1.12 s** |
| 1.85 s | 16,384 (late) | 2.44 s | 16,384 (late) | 2.41 s |
| 2.40 s | 16,384 | 1.97 s | 16,384 | 1.98 s |
| 2.95 s | 16,384 | 1.43 s | 16,384 | 1.42 s |
| 3.50 s | 16,384 | 0.84 s | 16,384 | 0.84 s |
| 4.05 s | never (late) | 5.55 s | **24,576** | **2.08 s** |
| 4.60 s | never (late) | 4.60 s | **24,576** | **1.51 s** |
| 5.15 s | never (late) | 4.03 s | **24,576** | **0.98 s** |
| 5.70 s | no boundary left | 3.09 s | no boundary left | 3.18 s |
| 6.25 s | no boundary left | 3.56 s | no boundary left | 3.54 s |

One stream decoding beside the read:

| sent after | main: gave way at | main: its TTFT | with: gave way at | with: its TTFT |
|---|---|---:|---|---:|
| 0.30 s | 16,384 (late) | 4.88 s | **8,192** | **2.09 s** |
| 1.05 s | 16,384 (late) | 4.11 s | **8,192** | **1.29 s** |
| 1.80 s | 16,384 (late) | 3.27 s | 16,384 (late) | 3.27 s |
| 2.55 s | 16,384 | 2.70 s | 16,384 | 2.73 s |
| 3.30 s | 16,384 | 1.94 s | 16,384 | 1.84 s |
| 4.05 s | 16,384 | 1.11 s | 16,384 | 1.13 s |
| 4.80 s | never (late) | 8.13 s | **24,576** | **3.15 s** |
| 5.55 s | never (late) | 7.59 s | **24,576** | **2.35 s** |
| 6.30 s | 32,768 (late) | 4.40 s | **24,576** | **1.57 s** |
| 7.05 s | never (late) | 6.31 s | **24,576** | **0.84 s** |
| 7.80 s | no boundary left | 5.70 s | no boundary left | 5.66 s |
| 8.55 s | no boundary left | 4.88 s | no boundary left | 4.88 s |

The two runs at each delay were within 0.1 s of each other, except the 7.66 s. That was the first read after that
engine start: its first chunk took 7.3 s instead of 1.9 s. Its `BYIELD` was out 6.7 s before the boundary and was
taken there.

</details>

**On 2x RTX 5090 with #2050** (`--batch 6 --batch-groups 2`, `STRATA_PIPED_READ_PIECE=8192`, the 0.1.43 server with
and without this change): with one-chunk pieces, 6 of 12 runs gave way a piece late without it and none with it. The
short request's TTFT went from 2.5-3.75 s to 0.94-2.05 s at those delays.

**Try it without a GPU** (the fake engine in `serve/test_parallel.py`, one boundary in the read):

```
python -m unittest serve.test_parallel -k far_apart
```

On main, the solo and admission tests fail in every run, 10 of 10 and 5 of 5, with `[] != [2560]`: the read finished
without giving way. With this change they passed 25 of 25 runs each, about 3 s per run.

**Where main is better:** I found no case. Where both gave way at the same boundary, or no boundary was left, both
requests' TTFTs were the same. Where main never gave way, the long prompt's own first token came 0.70 s later with
nothing decoding (0.34 s sooner with a stream decoding). That is #656's trade, which main makes too when the ask is in
time: the short request goes first. The poll's cost is one look at the waiting list (a lock and a scan) per half
second of engine silence during a read.

## What changed
- `serve/server.py`: `_control(..., on_quiet=None)` calls `on_quiet()` on its quiet poll. The solo and admission
  reads' give-way checks move, unchanged, into two small functions (`give_way()`, `ask()`). They run at a `PP` line, as
  before, and on that poll.
- `serve/test_parallel.py`: the fake engine takes `--pp-every K` (a `PP` line and a boundary every K chunks). Three
  tests: a solo read and an admission give way at their first boundary, and a request about as long gets no `BYIELD`
  on the poll either.

## Extra Notes
**Tests:** every `serve/test_*.py` module passes on main (`a0116b27`, 725 tests) and with this commit (728: the
three new ones). `serve.test_parallel` passed 3 of 3 full runs.

**How the one-GPU runs were timed:** a measurement-only patch (not in this PR) made both servers log each control
event: its time and token counts, no text. From that log, each run records when the short request started waiting,
when each boundary's `PP` line came, when the `BYIELD` went out and where the read gave way. The prompts were this
repository's own files, so every read was cold. I can share the scripts.

**Base:** rebased onto main `a0116b27`. The only change near it on main is #1994's `try`/`except` around the
admission loop, which now wraps the new `for` line. The #2050 numbers were measured on 0.1.43 with the same
`server.py` change.

**Not measured:** AMD and Intel GPUs. This is server only, with no engine or backend file changes, so there is no
CUDA, HIP or SYCL build to check. The gain grows with the chunk time: a late give-way costs one chunk, or the rest of
the read.