#2518 Concurrent requests break prefix cache

本文

Concurrent prefill appears to break prefix cache use.

## Reproducer

`llama-benchy` run with `--concurrency 2` against a Strata server running `--batch 2` never uses prefix cache.

Note: using `llama-benchy` against Strata requires this patch: https://github.com/eugr/llama-benchy/pull/33

## Correct Behavior with `--concurrency 1`

Here `llama-benchy` is run with `--concurrency 1`, performance is as expected for a cached prefix. And the monitor shows the prefix reused.

```
llama-benchy --base-url http://***:8081/v1 --model qwen38-flash-n
ext   --tokenizer Qwen/Qwen3.8-Flash-Next --depth 8192 --tg 128 --runs 1 --pp 2048 --concurrency 2 --enable-prefix-caching   --exit-on-first-fail --latency-mode generation --no-warmup
```

| model             |           test |            t/s |     peak t/s |   ttfr (ms) |   est_ppt (ms) |   e2e_ttft (ms) |       
|:------------------|---------------:|---------------:|-------------:|------------:|---------------:|----------------:|       
| qwen38-flash-next | ctx_pp @ d8192 | 1087.24 ± 0.00 |              | 5.59 ± 0.00 | 7536.53 ± 0.00 |  7538.09 ± 0.00 |       
| qwen38-flash-next | ctx_tg @ d8192 |   41.84 ± 0.00 | 42.00 ± 0.00 |             |                |                 |       
| qwen38-flash-next | pp2048 @ d8192 |  733.33 ± 0.00 |              | 6.84 ± 0.00 | 2792.72 ± 0.00 |  2794.29 ± 0.00 |       
| qwen38-flash-next |  tg128 @ d8192 |   47.25 ± 0.00 | 48.00 ± 0.00 |             |                |                 |       

### Strata Monitor Output

Strata Monitor shows that the prefix is reused as expected.

TIME|	STATUS|	PROMPT|	REUSED|	OUTPUT|	PREFILL T/S|	DECODE T/S|	VRAM HIT RATE|	DURATION
-|-|-|-|-|-|-|-|-
05:04:01 PM|	Max tokens|	10,242|	8,183|	128|	721|	44.1|	87.4% +3.7% PCIe|	5.8 s
05:03:50 PM|	Max tokens|	8,194|	0|	128|	1,067|	39.1|	89.1% +2.8% PCIe|	11.0 s

## Incorrect behavior with `--concurrency 2 `

Here `llama-benchy` is run with `--concurrency 2`, prefill performance tanks for what should be a cached prefix. And the monitor shows no prefix use.

```
llama-benchy --base-url http://***:8081/v1 --model qwen38-flash-n
ext   --tokenizer Qwen/Qwen3.8-Flash-Next --depth 8192 --tg 128 --runs 1 --pp 2048 --concurrency 2 --enable-prefix-caching   --exit-on-first-fail --latency-mode generation --no-warmup
```

| model             |                test |   t/s (total) |       t/s (req) |     peak t/s |   peak t/s (req) |    ttfr (ms) |       est_ppt (ms) |      e2e_ttft (ms) |
|:------------------|--------------------:|--------------:|----------------:|-------------:|-----------------:|-------------:|-------------------:|-------------------:|
| qwen38-flash-next | ctx_pp @ d8192 (c2) | 978.87 ± 0.00 | 794.07 ± 304.61 |              |                  | 15.31 ± 0.79 | 12099.93 ± 4640.99 | 12101.58 ± 4640.99 |
| qwen38-flash-next | ctx_tg @ d8192 (c2) |  18.64 ± 0.00 |    19.84 ± 9.38 | 41.00 ± 0.00 |     20.50 ± 9.50 |              |                    |                    |
| qwen38-flash-next | pp2048 @ d8192 (c2) | 188.50 ± 0.00 |  149.95 ± 55.69 |              |                  | 25.93 ± 1.18 | 15843.08 ± 5883.94 | 15844.73 ± 5883.94 |
| qwen38-flash-next |  tg128 @ d8192 (c2) |  15.82 ± 0.00 |   19.20 ± 10.39 | 39.00 ± 0.00 |    19.50 ± 10.50 |              |                    |                    |

### Strata Monitor Output

Strata Monitor shows no prefix reuse. The top two requests should re-use the prefix from the bottom two.

TIME|STATUS|	PROMPT|	REUSED|	OUTPUT|	PREFILL T/S|	DECODE T/S|	VRAM HIT RATE|	DURATION
-|-|-|-|-|-|-|-|-
04:48:22 PM|	Max tokens|	10,241|	0|	128|	889|	31.9|	91.1% +2.1% PCIe|	26.0 s
04:48:22 PM|	Max tokens|	10,242|	0|	128|	1,034|	9.0|	95.8% +0.2% PCIe|	24.3 s
04:48:01 PM|	Max tokens|	8,194|	0|	128|	907|	31.3|	91.3% +2.2% PCIe|	21.1 s
04:48:01 PM|	Max tokens|	8,195|	0|	128|	1,105|	10.6|	96.2% +0.2% PCIe|	19.6 s