#2986 Community benchmark: 2x RTX 5060 Ti 16 GB, Threadripper 3970X, Flash-Next IQ3_S (0.1.44)

本文

Adds a community benchmark report to `bench/results/` and an index line to `docs/COMMUNITY_BENCHMARKS.md`.

**Setup:** 2x RTX 5060 Ti 16 GB on PCIe gen4 x8/x8 (ASRock Rack TRX40D8-2N2T), Threadripper 3970X (AVX2 only, no AVX-512), 128 GB quad-channel DDR4-2666, Ubuntu Server 24.04 headless. Flash-Next **IQ3_S** at 262K context with KV streaming, layer split 25, `--pcie-frac 0.20`.

**Why this report:** the only existing 2x5060 Ti IQ3_S report (2026-10-02) ran 0.1.34 on PCIe gen3 with an 8-core desktop CPU and reached 56-63 tok/s decode; the newer rerun reports reach ~85 but with the lighter IQ3_XXS. This box shows IQ3_S on two 5060 Ti reaching that decode level when the cards get gen4 and a strong AVX2 CPU pool.

**Numbers (suite 1, 3 runs each):** decode 93.6 tok/s short chat, prompt 1185 / 2002 tok/s at 4K / 16K fresh, agent session 90.4 tok/s with 68.8 % conversation-cache hit.

**Extras that may be useful beyond the numbers:**
- A/B: `--prefill auto:32768` is **worse** on a two-card layer split (16K fresh 1531 vs 1895 tok/s; the big chunk borrows the second card's expert-cache slots) — the +21-35 % community reports find for it apply to single-GPU setups.
- A/B: layer split 25 slightly ahead of 20 on this box.
- Measured cost of `"parallel": 2` on 16 GB cards: 0.51 GiB of VRAM per slot per card, expert caches 5106/4393 -> 4528/3883 slots (-11 % per card).
- ESP (experimental speed projection) is ON in all runs, flagged in the report (community reports do not use it).

Speed suite only — no needle/recall check was run.