#3580 [Engine] Multi-turn answers shrink to a few tokens with --batch 8+ and a large --conversation-cache (0.1.40.4 and 0.1.45)

Beschreibung

### What happened

**Summary.** In one conversation that keeps adding turns, each answer is shorter than the last, until the engine stops after about 4 tokens on every turn. This happens only when the batch slots are 8 or more AND the parked-conversation cache is large (16384 MiB, 16 slots). With either setting alone, answer lengths stay normal. Prompt reuse looks healthy throughout (`prompt N tokens = X reused + ~320 read`, drafts accepted), so the engine itself ends the answers early.

**Load.** One stream for 150 s, one append-only conversation: a 1,500-token first turn, then +300-token turns, each asking for five bullet points; `max_tokens` 512; temperature 0.7, top_p 0.8, top_k 20, presence_penalty 1.5.

**Results (median answer length on turns 2 and later, and answers of 8 tokens or fewer):**

| parallel | conversation cache | engine | later turns | answers <= 8 tokens |
|---:|---|---|---:|---:|
| 4 | 3072 MiB / 4 slots | 0.1.40.4 | 220-237 | 0 |
| 8 | 3072 MiB / 4 slots | 0.1.40.4 | 212 | 0 |
| 4 | 16384 MiB / 16 slots | 0.1.40.4 | 209 | 0 |
| 8 | 16384 MiB / 16 slots | 0.1.40.4 (also with `--ple-io ram`) | 2-4 | 47 of 57 |
| 8 | 16384 MiB / 16 slots | 0.1.45 | 5 | 48 of 66 |
| 16 | 16384 MiB / 16 slots | 0.1.45 | 2 | 43 of 51 |
| 16 | 16384 MiB / 16 slots + STRATA_SLOT_COPY_DIRECT/BATCHSYNC/CKPT_MOVE/CKPT_POOL/BATCH_SELF_COMMIT/QSA_SLOTS | 0.1.45 | 39 | 0 (shrinks, no collapse) |

The answers shrink from turn 2 on, for example 512, 95, 60, 40, 47, 35, 26, 9, 8, 13, 9, 4, 4, ... (0.1.40.4, parallel 8, 16 GiB/16). With 4 or more concurrent streams, most later turns stay normal (one run at 8 streams, 0.1.45 parallel 8, did shrink). A Qwen3.8-27B on vLLM under the same load keeps its later turns at 512 tokens.

**Guess.** Whatever the engine does with the live conversation between turns differs once the slot count and the parked cache are both large. Perhaps a park/restore or slot copy of the GDN state or turn checkpoints. The copy/checkpoint switches change how badly it shrinks.

**Repro artifacts available:** configs, engine logs (`strata serve: prompt ...` lines per turn), and the harness (speed.py load --streams 1 --start 1500 --turn 300 --max-tokens 512).

### Strata version, GPU, OS

- Strata **0.1.40.4** (6674a00) and the Strata engine of the PolyStrata tree at **0.1.45** (974552f); both reproduce it.
- 2x RTX 2080 Ti 22 GB (sm_75), layer split, NVIDIA driver 595.91.07, CUDA 13.2.
- Ubuntu, Linux 7.0.0-38-generic.
- Qwen3.8-Flash-Next-GSQ-RCO IQ3_XXS (native shards), 2x RTX 2080 Ti 22 GB (sm_75), CUDA 13.2, layer split 24, `--kv int8 --kv-resident 32768 --max-context 131072 --spec 4 --mtp <rt> --resident-experts --expert-cache auto`, the server's `"parallel"`. Engines: Strata 0.1.40.4 (6674a00) and the Strata engine of the PolyStrata tree at 0.1.45 (974552f).

### Engine log

Per-turn lines from one single-stream run (0.1.40.4, parallel 8, 16384 MiB / 16-slot conversation cache). Prompt reuse stays normal while the generated length shrinks:

```
strata serve: prompt 1354 tokens = 0 reused + 1354 read in 4745 ms (285.3 tok/s), 512 generated in 7225 ms (70.9 tok/s), drafts accepted 274 of 356, 1 checkpoints
strata serve: prompt 2189 tokens = 1347 reused + 842 read in 3110 ms (270.8 tok/s), 95 generated in 1373 ms (69.2 tok/s), drafts accepted 51 of 66, 2 checkpoints
strata serve: prompt 2601 tokens = 2283 reused + 318 read in 2023 ms (157.2 tok/s), 60 generated in 924 ms (65.0 tok/s), drafts accepted 29 of 40, 3 checkpoints
strata serve: prompt 2971 tokens = 2660 reused + 311 read in 2090 ms (148.8 tok/s), 40 generated in 620 ms (64.6 tok/s), drafts accepted 17 of 28, 4 checkpoints
strata serve: prompt 3301 tokens = 3010 reused + 291 read in 1986 ms (146.5 tok/s), 47 generated in 657 ms (71.5 tok/s), drafts accepted 26 of 34, 5 checkpoints
strata serve: prompt 3685 tokens = 3347 reused + 338 read in 2212 ms (152.8 tok/s), 35 generated in 476 ms (73.5 tok/s), drafts accepted 20 of 28, 6 checkpoints
strata serve: prompt 4031 tokens = 3719 reused + 312 read in 2055 ms (151.8 tok/s), 26 generated in 410 ms (63.4 tok/s), drafts accepted 10 of 17, 6 checkpoints
strata serve: prompt 4381 tokens = 4056 reused + 325 read in 2185 ms (148.8 tok/s), 9 generated in 130 ms (69.2 tok/s), drafts accepted 5 of 9, 6 checkpoints
strata serve: prompt 4751 tokens = 4389 reused + 362 read in 2272 ms (159.3 tok/s), 8 generated in 107 ms (74.8 tok/s), drafts accepted 5 of 6, 6 checkpoints
strata serve: prompt 5110 tokens = 4758 reused + 352 read in 2267 ms (155.2 tok/s), 13 generated in 165 ms (79.0 tok/s), drafts accepted 8 of 9, 6 checkpoints
strata serve: prompt 5454 tokens = 5122 reused + 332 read in 2075 ms (160.0 tok/s), 9 generated in 182 ms (49.4 tok/s), drafts accepted 3 of 8, 6 checkpoints
strata serve: prompt 5785 tokens = 5462 reused + 323 read in 2190 ms (147.5 tok/s), 4 generated in 46 ms (87.5 tok/s), drafts accepted 2 of 2, 6 checkpoints
strata serve: prompt 6108 tokens = 5788 reused + 320 read in 2186 ms (146.4 tok/s), 5 generated in 86 ms (57.9 tok/s), drafts accepted 2 of 3, 6 checkpoints
```