#2989 batch-groups: a prompt with pictures is read beside the decoding slots too, stall 6.0 -> 0.1 s (opt-in)

说明

## Summary

With `STRATA_PIPED_CONCURRENT=1` (#2326, in 0.1.45) a text prompt is read on a thread of its own while the other slots keep decoding, but a prompt with pictures is still read the usual way, so every slot stops for it ([BATCHING.md](https://github.com/Niko1221/Strata/blob/main/docs/BATCHING.md): "A prompt with pictures is read the usual way"). This reads it the same way as a text prompt. On 2x RTX 5090, while a 50K-token prompt with a picture is read beside four decoding streams, the streams keep 34-37 tok/s instead of 4.5-6 tok/s, and their longest pause goes from 6.0 s to 0.1 s. Without `STRATA_PIPED_CONCURRENT=1` nothing changes.

## Motivation

Agents send screenshots, and a conversation with pictures is read again whenever its saved state no longer matches. In four days of our engine logs, 123 prompts with pictures were read the usual way. The longest was 190,489 tokens (25.2 s at 7,131 tok/s), and the other slots got 2.1 s of decoding in that time, 6-7 tok/s each.

## How it works

Pictures were kept out for one reason: a picture's rows go up through the prompt path's pinned slots (`core::PinnedRing`, #1996), whose size was taken to be known only at the upload. The read's thread could then have grown a slot, and a pinned allocation takes the driver's lock (#31). But that size is bounded: the gather sends a picture's rows in runs of at most 4 MiB.

- `prompt_uploads.hpp` (new, header-only arithmetic) holds the rule. `picture_run_rows()` is the gather's run (the same expression it used before), and `prompt_upload_bytes()` is the largest upload a chunk can send, pictures included.
- Before a concurrent read, `Prefill::reserve_uploads(T, pictures)` grows the ring to that, as the concurrent read already does for a text chunk. Only the first stage takes picture rows; later stages keep the text size.
- A prompt with pictures does not give way to a shorter request (`give_way`, as before). The engine now also ignores a `BYIELD` during its read (with a log line) instead of stopping a run that would not be resumed.
- `STRATA_PIPED_CONCURRENT_PICTURES=0` keeps the usual way, for an A/B.

Nothing else about pictures changes. The image-position table is written at the admission, before the read, as before. The embeddings, the conversation cache's picture keys and the vision encoder are untouched.

## How to review

Read `prompt_uploads.hpp`, then `Prefill::reserve_uploads`, then the three hunks in `generate.cpp` (the gate, the reservation, the `BYIELD`). The rest is the test and one bullet in `docs/BATCHING.md`.

## Results

2x RTX 5090 (Windows 11), Swift 1.5 IQ3_S, `--layer-split 25 --batch 6 --batch-groups 2 --resident-experts --kv-resident 32768`, the vision encoder on GPU 0. Engines built from 0.1.45 (main `477ac13c`) and from this branch, a fresh engine per arm. A cold prompt of about 49,800 tokens with one picture, read alone and then beside four streams that are decoding, twice per arm, temperature 0:

| | 0.1.45, default (D0) | this PR, default (D1) | `CONCURRENT=1`, `PICTURES=0` (P0, P0b) | `CONCURRENT=1`, this PR (P1, P1b) |
| --- | ---: | ---: | ---: | ---: |
| picture prompt alone, first token | 6.50 s | 6.50 s | 6.47, 6.49 s | 6.48, 6.47 s |
| beside four streams, first token | 6.42 s | 6.38 s | 6.37-6.42 s | 7.40-7.48 s |
| each stream during the read | 5.0-5.3 tok/s | 5.0-5.2 tok/s | 4.5-6.1 tok/s | 33.9-36.8 tok/s |
| longest pause of a stream | 5.95 s | 5.96 s | 5.91-6.01 s | 0.07-0.16 s |

Order: D0, D1, P0, P1, P1b, P0b. The engine logged each concurrent picture read as about 49,830 tokens in 6.95-7.14 s, with the slots decoding 128-143 rows/s meanwhile. P1b then read six more picture prompts beside the four streams (7.40-7.89 s each to the first token), with no stall or error lines.

**The default path.** Only one line runs differently there: the gather's run size, which `batch_read_test` checks equals the old expression for every row width up to 8 MiB (the rest is behind `STRATA_PIPED_CONCURRENT`). Without `STRATA_PIPED_CONCURRENT`, 0.1.45 and this build gave the same picture answer beside the streams, at the same time to first token (D0, D1 above).

**Exactness.** The picture prompt's answer in the first round beside the streams was the same in all six arms. In the second round it was the same in P0b, P1 and P1b, and differed in P0, which reads pictures the usual way with the same settings as P0b. So the usual path itself varies there from run to run, as batch answers can; reading the picture concurrently did not change it. The four streams' own answers differ from run to run on any build, so they are not a test of this.

**On our server, before 0.1.45.** The same change on 0.1.44.1 + #2050 + #2326, live traffic around it, a cold 49,959-token prompt with one picture joining two decoding streams: each stream kept 49-52 tok/s instead of 12.6-13.8 tok/s with #2050's 32K pieces, the longest pause went from 3.9 s to 0.1 s, and the prompt's time to first token was 7.4 s against 7.5-7.6 s. On 0.1.44.1 + #2326 alone, beside four streams: 34.9-37.0 against 4.9-5.9 tok/s, pause 0.07-0.13 s against 6.0 s.

## Costs and limits

- The prompt with pictures itself waits about 1 s longer for its first token when slots are decoding beside it (7.4-7.5 s against 6.4 s for 50K tokens beside four streams, above): its read shares the GPU with them. Read alone, it is unchanged.
- Pinned RAM: the ring's 4 slots grow to a picture run (409 rows of 10,240 bytes, about 4 MiB each), so about 16 MiB on the first device, once, the first time a prompt with pictures is read this way.
- A prompt with pictures still never gives way, so a short request that arrives during its read waits for it, as before. The difference is that the decoding slots no longer wait too.
- The image-position table is one per device, shared by all slots (#1242 is about that). This PR leaves it as it is.

## What changed

- `include/strata/prefill/prompt_uploads.hpp` (new): `kPictureRunBytes`, `picture_run_rows()`, `prompt_upload_bytes()`.
- `src/prefill/prefill.cpp`: the picture gather takes its run from `picture_run_rows()` (same value); `reserve_uploads(T, pictures)` sizes the ring with `prompt_upload_bytes()`.
- `include/strata/prefill/prefill.hpp`: `reserve_uploads(int64_t T, bool pictures = false)`.
- `src/program/generate.cpp`: a prompt with pictures takes the concurrent read unless `STRATA_PIPED_CONCURRENT_PICTURES=0`; the reservation covers its pictures; a `BYIELD` during its read is not taken.
- `docs/BATCHING.md`: the bullet that said pictures are read the usual way.
- `tests/program/batch_read_test.cpp`: the reservation's sizes, that every upload the picture gather makes fits the reserved slots (over chunks with picture rows in different places), and that the gather's run is unchanged.

## Extra Notes

- **Builds:**
  - CUDA on Windows (MSVC 2022, CUDA 13.0, sm_120).
  - HIP on Windows (gfx1100): the engine and `batch_read_test`. A full HIP build with tests stops at 0.1.45's own `src/kernels/hc_persist_parity.cpp` (`setenv` / `unsetenv` on Windows), which this PR does not touch.
  - Linux (gcc 13.3, CUDA 13.0 headers): `prefill.cpp` and `generate.cpp` compile. On 0.1.44.1 the whole Linux engine was built (WSL2 Ubuntu 24.04, sm_120).
  - `batch_read_test` passes on Windows (CUDA and HIP) and Linux. Its new checks catch a reservation that leaves pictures out, a run one row too long, and a run sized in token ids (each mutant tried).
- **Mac:** the two shared files this touches compile in the Metal configuration (the Mac build's include order and defines, checked with g++). The Mac build of 0.1.45 itself also needs #2984.
- **SYCL:** the port compiles its own `generate.cpp` and `prefill.cpp` (`sycl/src/`), which this PR does not touch. From `prefill.hpp` it only gets a defaulted parameter on a member it neither defines nor calls. Not built: no oneAPI here.
- **Not measured:** AMD on a GPU (built only), Linux on a GPU (built and unit-tested only), `--mtp` with pictures (the drafter's prompt pass does not read picture rows), `--batch-groups 1`, and Mac (no concurrent read there).