#2973 Layer split + `--batch`: slots decode without MTP drafts, and `--batch-mtp` is slower on 4 stages. Could pipelined groups draft?

Descripción

## Summary

On a 4-GPU layer split, a request alone decodes fast on the solo path (MTP + suffix drafts), but as soon as several requests share the batch slots, each one decodes without drafts (the pipelined `--batch-groups` path has none). `--batch-mtp --batch-groups 1` brings one draft per slot back, but the serial windows cost more than the drafts gain: 4 concurrent streams lose 30-40% of their total throughput against the pipelined default.

The gap is largest on copy-heavy work, which is most of what coding agents do: a file edit decodes at 174 tok/s alone with 98% of drafts accepted, and at 44-69 tok/s per stream when four run together.

Would it be possible for the pipelined groups to run MTP drafts per slot (even one proposal per slot per window), so a 4-stage split can pipeline and draft at the same time?

## Setup

- Strata 0.1.45, `main` at `477ac13c`, built from source by setup (sm_86, CUDA 13.2), Linux (Debian 13), driver 595.71.05
- 4x RTX 3090 24 GB, each on PCIe 3.0 x8; 2x Xeon Gold 6126 (2 NUMA nodes); 376 GB RAM
- Unsloth UD-Q4_K_XL (revision `38bb39e`), every expert in RAM, 94% of experts resident in VRAM, decode expert-cache hit rate 100%
- `"parallel": 4` (`--batch-groups auto`: 4 groups of 1 slot), `"layer_split": "11,24,36"`, `--trim-stage-weights`, `--max-context 262144`, `--kv fp16 --kv-resident 32768`, `--spec 4 --spec-min-p 0.70 --pcie-frac 0.35` (the last two from `--calibrate`), `--ple-io ram`, `--conversation-cache-mib 65536 --conversation-cache-slots 32`, `--vram-reserve-later-mib 1300`
- env: `STRATA_PIPED_CONCURRENT=1 STRATA_MMQ_KQ=1 STRATA_HC_Q8=1 STRATA_ROUTE_TAIL_SKIP=0 STRATA_SLOT_COPY_DIRECT=1 STRATA_SLOT_COPY_BATCHSYNC=1 STRATA_CKPT_POOL=1 STRATA_QSA_SLOTS=1`

## Measured

Requests to `/v1/chat/completions`, `reasoning_effort: none`, 900-token answers, a random tag at the start of every prompt so nothing is reused. Three workloads: write a new Python module (code), return an 11 KB PHP file with one method renamed (edit), write a short story (prose). "1 user" is the mean of two sequential requests; "4 users" is four concurrent requests, total output tokens over wall time, with each stream's own decode rate in brackets. A fresh engine for each arm; one round per arm.

| tok/s | code: 1 user / 4 users | edit: 1 user / 4 users | prose: 1 user / 4 users |
| --- | --- | --- | --- |
| default (pipelined groups, no slot drafts) | 139 / 209 (56-62 each) | 174 / 129 (44-69 each) | 95 / 210 (56-62 each) |
| `--batch-mtp --batch-groups 1` | 141 / 144 (38-41 each) | 177 / 99 (28-40 each) | 97 / 127 (33-34 each) |

Draft acceptance on the solo path: code 85-87%, edit 98%, prose 63-65%.

For completeness, on the same rig and workloads, none of these moved the 4-stream totals beyond run-to-run noise: `--spec 3` (191 / 122 / 193), `--spec 5` (203 / 129 / 212), `--lookup-chain 4` (192 / 121 / 201).

## Why it matters here

The workload is coding agents: 1-4 requests at once about 99% of the time (14 days of request logs), most turns either writing code or returning files with small edits. Those are exactly the requests where the solo path's drafts pay most (98% acceptance on edits) and where they disappear as soon as a second agent is busy.

`docs/BATCHING.md` notes `--batch-mtp` has only been run on two stages; on four it looks like the serial windows leave three of the four cards waiting, which outweighs the drafts.

Happy to test a branch on this rig and report numbers in the same format.