#1999 serve: a client that leaves during an image encode cannot be cancelled — prepare() runs before the cancel event, and the encode holds the single request turn

本文

 ### What happened

## Summary

`Service.prepare()` downloads image URLs and runs the image encoder **before** the request's
`cancel` event exists, so a client that hangs up during preparation cannot be cancelled: the
encode runs to completion for a request nobody is waiting for, and — because the encode takes
`self.fifo` — the engine sits idle for the whole encode while other requests queue behind it.
With `"vision": {"gpu": false}` the FIFO is not needed for the GPU-contention reason at all.

## Environment

Strata 0.1.43 (`7695859a`), Ubuntu 24.04.5, 2× RTX 5070 Ti 16 GB (layer split 6, one card in a
chipset slot that runs at PCIe 3.0 ×2), `serve/server.py --engine strata`, model Qwen3.8-Flash-Next
IQ3_S, `strata-iq3_s.json`: `"vision": {"gpu": false, "max_tokens": 1024, "threads": 8}`,
`--vision`, layer split 6, no `--batch` (one request at a time).

## What the code does

`prepare()` takes the engine's turn for the encode (`serve/server.py:3484`, the comment above it says
it is there so the engine and the encoder are never on the GPU at the same time), and it also
fetches image URLs (`3449`, `3478`, `urllib` with a 60 s timeout each). All of this happens before
any cancel event exists:

```text
serve/server.py:5366   ids, thinking, max_new = svc.prepare(...)     # downloads + encodes here
serve/server.py:5368   cancel = threading.Event()                    # only now
serve/server.py:5369   self._watch_client(cancel)                    # #430 #431: the disconnect watcher
```

the same order in `_responses_prepare` (`5547` → `5590`) and `_anthropic` (`5588` → `5590`).
`prepare()` has no `cancel` parameter, and `Vision.encode_all` / `encode` never look at one.
So #1619 ("give the place up when cancelled while queued") and #430/#431 ("a request is cancelled
within a second of its client hanging up") both start where `prepare()` has already finished.

## Measured

A 768×768 random-noise PNG (unique, so nothing is cached) sent to `/v1/chat/completions`
with `max_tokens: 1`, client aborted after 2 s; a plain text request sent ~0.6 s later;
`/proc/<pid>/stat` sampled every 0.5 s, `/status` alongside. Two runs:

| | run 1 | run 2 |
| --- | ---: | ---: |
| text request alone (baseline) | 0.90 s | 0.82 s |
| the same text request, sent 0.6–1.2 s after the client left | 2.92 s | 3.50 s |
| blocked by | **+2.02 s** | **+2.68 s** |
| `strata-vision` CPU after the client was gone | 41.6 cpu·s | 41.8 cpu·s |
| engine CPU during the same window | 0.0 cpu·s | 0.0 cpu·s |
| `/status` during the window | `busy: false, queued: 1` | `busy: false, queued: 1` |

The encoder is still working ~3.4 s after the client disappeared (8 threads, ~5 s wall for this
test picture), the engine does nothing at all during that time, and the next request waits for it.
A real screenshot costs ~11 s of CPU encode on this machine; #1404 measured 15.3 s for a
1024×1024 picture on a 9700X — so the idle engine window is that long, per new picture.

Two runs, one model, one picture size; the absolute numbers are this machine's.

## Why the FIFO part is worth separating

The comment in `prepare()` justifies the turn by GPU contention. With `"vision": {"gpu": false}`
the encoder is a separate process that never touches the GPU (the docs and #1404 recommend this
setup precisely to keep VRAM for the expert cache) — yet the turn is held for the whole encode,
and the engine goes idle. On a CPU encoder that looks like pure loss.

## Suggested minimal fix

1. Build `cancel` before `svc.prepare(...)` at all three call sites and pass it down:
   `prepare(..., cancel)`, `_note_unreadable_tool_images`, `Vision.encode_all`/`encode` —
   checking `cancel.is_set()` between images and between the URL downloads (each download can
   wait 60 s). A cancelled preparation ends the way the queued cancellation of #1619 does:
   `finish: "cancel"`, its own history/totals row, the request's embeddings file removed
   (`drop_embeddings()` already covers that).
2. Optionally, and separately: when the encoder is CPU-only, do not take `self.fifo` for the
   encode at all (or take it only around the part that uses the GPU), so a picture does not stop
   text requests for its whole encode.

## Not tested

### Strata version, GPU, OS
   Strata 0.1.43 (engine 0.1.43 per GET /v1/status) · 2× RTX 5070 Ti 16 GB · Ubuntu 24.04.5

### Engine log