#2643 Add community benchmark: RTX 4060 Ti 16 GB + RTX 3060 12 GB, IQ3_S — plus a layer_split sweep

Description

Community benchmark on an asymmetric dual-GPU pair, plus a `layer_split` sweep that may be worth upstream attention.

## Hardware

RTX 4060 Ti 16 GB + RTX 3060 12 GB (28 GiB total VRAM), Xeon E5-2696 v3 (18c/36t, AVX2, no AVX-512), 125.6 GiB DDR3, Linux 5.15, driver 595.91.07. Strata 0.1.44.1 (`296b59d2`), original Flash-Next IQ3_S, 262,144-token context, `--kv int8 --kv-resident 32768`, `--vision`, both cards at their **default** power limits.

## Results

Measured with the project's own `tools/strata_bench.py`, 3 runs per test, medians:

| Configuration | Prompt tok/s | Decode tok/s | TTFT |
| --- | --- | --- | --- |
| short chat | — | 50.4 (45.4-60.0) | 0.74 s |
| 4K prompt, fresh | 488.2 (486.5-488.2) | — | 8.27 s |
| 16K prompt, fresh | 835.0 (730.7-866.2) | — | 19.28 s |
| agent session, follow-up turns | see README | 36.2 (27.1-55.8) | 8.45 s |

Conversation cache hit over the agent session: 68.6 %.

## The finding: `layer-split auto` costs a lot of prefill on this pair

`auto` gave the **RTX 3060 3 of 48 layers**, leaving 12 GB of its VRAM idle and the card drawing 17 W through an entire prefill while the 4060 Ti sat at 100 % utilisation. Sweeping `layer_split` on an identical cold 14,564-token prompt, fresh server per point:

| `layer_split` | 3060 layers | 4060 Ti layers | prefill tok/s |
| ---: | ---: | ---: | ---: |
| `auto` | 3 | 45 | 452.6 |
| 12 | 12 | 36 | 581.8 |
| 18 | 18 | 30 | 693.0 |
| 24 | 24 | 24 | 761.7 |
| **28** | **28** | **20** | **802.1** |
| 32 | 32 | 16 | 803.2 |

**+77 % prefill at split 28**, plateauing at 28-32. Decode improved too (41.8 → 50.4 chat median; 25.1 → 36.2 agent median), since fewer layers per card lets the expert cache cover each layer better.

I understand from the log that the engine chose this deliberately — it cites avoiding a stage whose tiny expert cache would clamp the prompt chunk of every card (#1616). That caution is reasonable in general, but on a pair this asymmetric it costs 44 % of prefill throughput, and the clamping case did not materialise at any split I tested.

## Two things that may generalise

**A mis-set split shows up as a flat prefill-vs-length curve.** Under `auto`, prefill barely moved with prompt length (491.7 tok/s at 4K, 512.1 at 16K — no batching benefit). At split 28 it becomes a slope (488.2 at 4K, 835.0 at 16K). The 4K figure is unchanged, so a 4K-only benchmark cannot detect this at all.

**Memory bandwidth, not SM count or FP32 TFLOPS, tracked the optimum.** The 3060 has 360 GB/s against the 4060 Ti's 288 GB/s, making it the better host for expert-heavy layers despite fewer and slower SMs. The intuitive rule — give the *faster* card proportionally more layers — predicts the opposite direction, and I got it wrong twice before measuring.

## Also observed

The below-512 MiB VRAM warning (#2065) fires at the **low** split values, where the 4060 Ti carries 30-36 layers, and disappears at 24 and above — the tight card is not necessarily the smaller one. `--vram-reserve-mib` was raised 700 → 1024 alongside the split change.

## Limitations

Speed only: no correctness, needle or concurrency tests. The sweep points are single cold-prompt runs; the suite results are 3 runs. Absolute throughput drifts roughly 10-15 % with server age (expert profiles adapt after start), so the sweep is comparable within itself rather than against unrelated runs.

Full per-request data is in `results.json`. Note that `layer_split` is a top-level config key rather than an `args` entry, so it does **not** appear in the engine command line recorded in the report — the README states the split explicitly for that reason.