#2099 serve: sub-agent fan-out gets 0% sibling prefix reuse without --prompt-cache-tail - 93% and 11.7x prefill with it (GTX 1080 Ti, IQ3_S, 0.1.43)

说明

## TL;DR

Sub-agent requests that share a long prefix get **0% reuse on defaults** when the prompt is below the first periodic checkpoint (16,384): each sibling re-reads everything (~143 s per 14.8K-token prompt on this box). `--prompt-cache-tail` alone takes sibling reuse to **93% / 11.7x faster prefill**; `STRATA_CACHE_MESSAGE_BOUNDARY=1` takes the agent retry pattern to **3.1x**; `--prompt-cache-every 4096` recovers 33-65% for mid-prompt forks. All measurements, including two negative results (`"parallel": 2` does not fit an 11 GB card at 262K context; `--prompt-cache 10` does not widen the eviction window), are in the body and the follow-up comments below. Ask: consider `--prompt-cache-tail` on by default for `--serve`, and a pre-start check that `--batch N` at a given `--max-context` leaves the verify tier enough expert slots.

## Index of the follow-up measurements (comments)

1. [Retry/edit-last-message with `STRATA_CACHE_MESSAGE_BOUNDARY=1`](https://github.com/Niko1221/Strata/issues/2099#issuecomment-6123564966) - 99.6% reuse, 3.1x
2. [`"parallel": 2` negative result + pre-start check suggestion](https://github.com/Niko1221/Strata/issues/2099#issuecomment-6124240821) - slot sessions cost 0.95 GiB each at 262K context
3. [Mid-prompt forks + `--prompt-cache-every 4096`](https://github.com/Niko1221/Strata/issues/2099#issuecomment-6125105162) - chunk geometry: checkpoints land on chunk boundaries
4. [Checkpoint eviction window = 4 intervening requests](https://github.com/Niko1221/Strata/issues/2099#issuecomment-6126212660) - the two caches are independent
5. [`--prompt-cache 10` negative result](https://github.com/Niko1221/Strata/issues/2099#issuecomment-6127089250) - the K=5 cliff is not governed by the checkpoint pool; chunk geometry is a per-boot lottery

---

## Expected / Actual

**Expected:** a request whose prompt shares a prefix with a recently served prompt reuses the shared part (that is what the bounded checkpoint cache is for).
**Actual:** with `--prompt-cache-every` at its default 16,384 and a prompt below that length, no checkpoint exists at all, so a sibling that diverges at the tail re-reads the entire prompt: measured 0% reuse, three times in a row (14.8K-token prompts, ~143 s each on this box).

## Goal

One coding-tool conversation that spawns sub-agents: sibling requests that share a long prefix (system prompt + repo context, ~14.8K tokens) and diverge only in a short task tail (~30 tokens). Measured on a real serve box; the numbers below show sibling reuse is 0% on defaults and ask whether `--prompt-cache-tail` (or an equivalent) should be considered for agent-style workloads by default.

## Environment

| | |
|---|---|
| Engine | 0.1.43, serve (OpenAI dialect), `--spec 4 --spec-min-p 0.70 --mtp`, `--kv int8 --kv-resident 32768`, `--pcie-frac 0.00` |
| GPU | GTX 1080 Ti 11 GB (sm_61, CUDA 12 engine), 100% util, `vram_free_mib` 1887 |
| CPU/RAM | Xeon E5-2696 v4 (22C/44T, 2 NUMA), 125 GB RAM, 43 pool workers |
| Model | Qwen3.8-Flash-Next GSQ-RCO IQ3_S |
| Fresh prefill speed | ~102-109 tok/s (experts from RAM) |

## Method

Three sibling POSTs to `/v1/chat/completions`, back to back, same context (14,800 tokens: engine + server sources and docs), three different task tails, temperature 0, `chat_template_kwargs {"enable_thinking": false}`, `max_tokens` 32, unique salt in the system message per run. `reused` / `prompt_read` / `prompt_ms` from `/metrics`. Run twice: once with defaults, once with `--prompt-cache-tail` (only change; engine restarted between runs).

## Baseline (defaults: `--prompt-cache` 6, `--prompt-cache-every` 16384, no tail)

| request | prompt_total | reused | read | prompt_ms | wall |
|---|---:|---:|---:|---:|---:|
| sibling 0 (first) | 14,795 | 0 | 14,795 | 143,152 | 143.4 s |
| sibling 1 | 14,801 | **0** | **14,801** | 143,111 | 144.4 s |
| sibling 2 | 14,800 | **0** | **14,800** | 131,180 | 143.4 s |

0% sibling reuse: the whole prompt is under the first periodic checkpoint (16,384), so no state exists to reuse, and each sibling re-reads everything. A 3-way fan-out costs ~7.2 min of pure prefill on this box. For contrast, the tool's own main conversation (a growing conversation, 159-163K prompts) reuses 99.9%+ and reads 16-304 tokens per request.

## With `--prompt-cache-tail` (the only change)

| request | prompt_total | reused | read | prompt_ms | wall |
|---|---:|---:|---:|---:|---:|
| sibling 0 (first) | 14,795 | 0 | 14,795 | 131,682 | 132.0 s |
| sibling 1 | 14,801 | **13,824** | **977** | 12,184 | 37.1 s |
| sibling 2 | 14,800 | **13,824** | **976** | 12,077 | 50.8 s |

93.4% reuse (13,824 = the chunk boundary), sibling prefill 11.7x faster, sibling wall 3.9x. The whole 3-way fan-out drops from ~431 s to ~220 s.

## Ask

Primary: consider `--prompt-cache-tail` on by default for `--serve`, or at least a pointer to it in BATCHING.md's agent/slot section - its cost is one extra checkpoint write per fresh long prompt, and for prompts below `--prompt-cache-every` it is the difference between 0% and 93% sibling reuse.

Secondary (happy to split into its own issue if easier to track): a pre-start check that `--batch N` at a given `--max-context` leaves the verify tier at least its 367 slots, before the ~4-minute model load - from the negative result in the comments.

## Reproduction

1. `serve` on any single-GPU config with the defaults (`--prompt-cache` 6, `--prompt-cache-every` 16384, no tail flag)
2. POST `/v1/chat/completions` with a ~14.8K-token prompt (system + concatenated sources + a one-line task), temperature 0
3. POST the same messages again with only the task line changed
4. Read `/metrics`: the second request shows `reused: 0`, `prompt_read` = the full prompt, `prompt_ms` ~= the first request's
5. Restart the engine with `--prompt-cache-tail` added; repeat 2-4: the second request now shows `reused` at the last chunk boundary (13,824 of 14,795 here) and ~12 s prompt_ms

## Related

- #2137 (found while measuring the PCIe tier for this series: the Monitor page showed "Gen3 · idle Gen1" while the link was downgraded to Gen1 the whole time)