#2085 Community Bench: 2026-10-11-community-rtx-5060ti-3060-split — RTX 5060 Ti 16 GB + RTX 3060 12 GB, unequal links, `--layer-split` planner + the two new 0.1.43 defaults

Description

Measured on 2026-10-11/12 (one continuous evening session, 22:39–00:46) by @mmbeanz, on "bertha": an **unequal 16+12 GB pair with unequal weak PCIe links** — RTX 5060 Ti at gen5 **x8** (measured H2D 28.2–28.7 GB/s) + RTX 3060 at gen3 **x4** behind the chipset (measured H2D 3.15–3.2 GB/s). IQ3_S GSQ-RCO, 512K context, engine `7695859a` (v0.1.43) source build, Linux/nspawn, driver 615.71.09, CUDA 13.4.

This answers **TEST_REQUESTS row 1** (two NVIDIA cards of different sizes, `--layer-split auto`) and tests the two live default debates (**#2010/#1975/#2011** overlap, tail-skip) on a **split** — where it turns out the overlap never runs at all (finding 4, corrected after review) while tail-skip does (finding 5).

Full report with per-run JSONs, `/metrics?requests=all` histories, engine logs and configs: **PR #2083** (`bench/results/2026-10-11-community-rtx-5060ti-3060-split/`).

## Speed suite (medians, 3 runs each, 0 failures)

| run | change vs baseline | prompt 4K | prompt 16K | prompt 32K | TTFT 32K | chat decode |
| --- | --- | ---: | ---: | ---: | ---: | ---: |
| A | baseline `--layer-split 40` | 868 | 1,413 | 1,721 | 18.7 s | 74.4 |
| B1 | `--layer-split auto` → chose K=3 | 836 | 1,227 | **1,291 (−25.0%)** | 24.9 s | 66.6 |
| B1c | explicit `--layer-split 6` (the engine's printed suggestion, followed literally) | 264 | 346 | 408 (−76%) | 78.6 s | 41.4 |
| B1d | explicit `--layer-split 42` (the as-given-frame equivalent) | 889 | 1,397 | 1,683 (−2.2%) | 19.1 s | 76.2 |
| C | `STRATA_ADAPT_OVERLAP=0` (inert on a split — see finding 4) | 868 | 1,413 | 1,723 (+0.1%) | 18.6 s | 75.3 |
| D | `STRATA_ROUTE_TAIL_SKIP=0` | 871 | 1,411 | 1,723 (+0.1%) | 18.6 s | 72.0 |

## What we found

1. **The auto planner's probes are right, its choice costs 25% of 32K prompt** on this pair: K=3 strands 3,548 MiB on the 3060 and clamps the prompt chunk to 3,328 (#1616) with 45 layers on the 5060 Ti. Its own caveat ("prices decode, not the prompt") was exactly right.
2. **The printed suggestion is in the wrong frame.** `auto` reorders cards so the faster one is last (#1352), but an explicit `--layer-split N` does not reorder — so its `--layer-split 6` literally means 42 layers on the slow card (B1c: −76%, hit_rate 0.669, ~7 GiB stranded on the 16 GB card). **Question for maintainers: should the suggestion be printed in the as-given frame, or should explicit splits inherit the reorder?**
3. **Hit rate is per-layer coverage, not held count.** A (8,842 experts held) vs B1 (5,935): statistically identical hit rate (0.859 vs 0.865, permutation p=0.93) despite 33% fewer held experts. Neither the old nor the new `H(f)` coverage curve survives this; what tracks is each stage caching its own layers' experts (B1c is that effect taken to the extreme).
4. **`STRATA_ADAPT_OVERLAP` is inert on layer splits** (C, corrected 2026-10-12 after review in #2010). The engine disables the overlap *before consulting the flag* (`generate.cpp`: `why = multi_gpu || !stages.empty() ? "not on a layer split"`; our `--remote-expert-opt` is a second gate), so our A/B ran the same configuration twice — the null result (prompt +0.14%, decode p=0.39, hit_rate p=0.92, `pcie_share` 0 on all 40 requests in both arms) is the signature of configuration equality, not a topology finding, and our original "does not reproduce on a split" headline is retracted. What stands: on a split the flag cannot cost anything because the feature never runs there — the −21%/−26% single-GPU regressions are structurally impossible on this topology. 0.1.43 logged nothing about the flag in either arm; fixed by #2011's state logging, shipped in 0.1.43.1. Thanks @Krypto-Whitehat for the code check.
5. **Tail-skip on a split: ~+5.2% decode, not +10–20%, and it decays with context**: +9.3% at 4K, +4.8% at 16K, **+1.0% at 32K** (means 71.10 vs 67.58, p=0.20); prompt unaffected; draft acceptance identical. Keep the default. Related to #2007 (a T10 split measured −1.4% for `=7`).
6. **`hit_rate` silently changes meaning with the tail-skip flag.** From our two logs: A's reported lookups/token (564.3) exclude the 98,217 tail-skipped misses; on a common denominator skip ON measures **0.752 vs 0.775** — the sign reverses. Worth a doc line for `/metrics`/Monitor consumers; and with the flag off the log prints nothing at all, so the off state is only inferable from absence.
7. **Correctness: needles 9/9** at 32K/128K/**256K** × depths 10/50/90, cold boot, defaults on (including tail-skip).
8. **Tool findings, confirming and extending #2062:** `gpu_slug()` drops the second card on mixed hosts (the maintainer reports the mixed-card slug fix as done in #2062, after 0.1.43); the remote `machine` block blends client fields (our runs recorded 3.8 GiB RAM and a null driver); the generated README cites `engine-log-excerpt.txt` in `--url` mode where no such file is written. Separately, see #2088: **`--expert-profile-save` writes on server QUIT, so a bench run that starts its own server from a config carrying that flag silently overwrites the live profile with what it learned.** Correction: in `--url` mode the bench only attaches and does not stop the server (per the maintainer in #2088); a fix that strips the flag from bench-started servers is in. We hit this; pin the profile and drop `--save` for benching.

## Not tested

Long soak (planned follow-up with RSS/VRAM sampling), batching/concurrency, vision latency, mmap file tier, Windows, HIP, single-card configs on this box, and the answer-quality cost of tail-skip beyond needles. `hit_rate` is the decode-phase metric only; permutation p-values treat requests as independent, which the adaptive tier correlates — they are optimistic.