#3321 Slot <-> pool copy bandwidth is ~5 GB/s (takes/gave back/park), ~19 GiB conversation costs ~11 s per agent turn; PCIe probe says 26.8 GB/s

说明

## Environment

- Strata 0.1.45, 4x RTX 4090 (PCIe, host<->device probe 26.8 GB/s), 566 GB RAM
- `--batch 4 --batch-groups auto --kv k8v4 --kv-resident 524288`, conversation cache 384 GiB / 192 slots
- Workload: long-context (300K-680K tokens) multi-turn agent traffic, 4-5 concurrent conversations

## Observation

With 5 concurrent conversations on 4 slots, a conversation is evicted back to the pool and re-admitted between turns. The per-turn log of a ~497K-token conversation:

```
slot 2 gave back 496668 tokens ... in 3701.7 ms
slot 1 takes 497051 tokens (copied in 3734.8 ms)
prompt 497051 tokens = 496668 reused + 383 read in 7992 ms
parked 497051 tokens in 3556.0 ms
```

The next turn is the starkest case: **`497429 tokens = 497428 reused + 1 read in 7923 ms`** - 99.9998% cache reuse, a single token actually read, and the turn still costs ~8 s, all of it the gave-back + takes copies.

Per turn the conversation pays:

| step | time | effective bandwidth |
|---|---|---|
| gave back (slot -> pool) | 3.7 s | 497K tokens x 816 B x 48 layers = ~19.5 GiB / 3.7 s = **~5.3 GB/s** |
| takes/copied (pool -> slot) | 3.7 s | ~5.3 GB/s |
| park (snapshot to pool) | 3.5 s | |

So every agent turn carries a **~11 s pool-shuttle tax** that the 99%+ prompt-cache reuse cannot recover (the reuse is prompt-side; the KV itself still shuttles). At 4-5 concurrent conversations slot rotation happens between most turns. A `restored ... in 152.9 ms` line shows the restore-metadata path itself is fast - the cost is the bulk KV copy.

The engine's own PCIe probe at start reports **26.8 GB/s host->device**, so the copy runs at ~1/5 of what the bus did in the probe. Plausible causes: a staging buffer that is not pinned, a single copy stream (no overlap of the per-stage/per-layer copies), or device-to-host-to-device hops instead of a direct path.

## Suggested directions

1. Pin the staging buffers and issue the per-layer/per-stage copies in parallel streams (the layer split already knows the per-GPU extents).
2. Where topology allows, copy pool <-> slot directly between the two device buffer sets (or through a pinned bounce buffer sized to a layer, not the whole conversation).
3. Even a 2-3x improvement would cut the tax from ~11 s to 4-5 s per turn; combined with more slots (so conversations stay resident and skip the shuttle entirely) the tax disappears for the common case.

Happy to test candidate patches on the 4-GPU box (the 5.3 GB/s number is stable and easy to reproduce: any 5-vs-4-slot rotation with a ~500K-token conversation).