#2163 bench: 2x RTX 3080 Ti 12 GB community results (Flash-Next Q4_K_M abliterated, 196K ctx, layer split)

Description

Community benchmark: **2x RTX 3080 Ti 12 GB**, Qwen3.8-Flash-Next **Q4_K_M abliterated** (community quant, 36-shard GGUF + native pack), **196,608-token context**, CUDA layer split. Run with the stock `tools/strata_bench.py` suite (`--runs 3 --long`) against the daily-driver server as recommended in `docs/COMMUNITY_BENCHMARKS.md`.

## Setup

| | |
|---|---|
| GPUs | 2x NVIDIA GeForce RTX 3080 Ti 12 GB (24 GiB total), driver 595.91.07, CUDA 13.2. Second card sits on a PCIe x4 link (probed 3.3 GB/s host→device vs 21.1 GB/s on the first) |
| CPU / RAM | i5-13600K (14c/20t, AVX2, no AVX-512), `performance` governor, 76.1 GiB RAM |
| OS | Linux 7.0.0-38-generic |
| Strata | engine 0.1.43 (source build `3ef5e6c5`), Linux CUDA sm_86 |
| Model | `Qwen3.8-Flash-Next-Abliterated-Uncensored-AD-4.25bpw-Q4_K_M`, PLE table 320M rows, expert blobs up to 2.66 MB, 52.59 GiB total |

Non-default serving flags (each A/B-measured on this box, kept only if it won and stayed stable):
- `--trim-stage-weights --layer-split 27` — the big one: each card loads only its layers' dense weights. Resident experts **4694 → 6714** (+43%), prompt chunk 7424 → 13312, decode +24%. `auto` picks K=41 here (7 layers on the slow x4 card) and decodes far worse, so a manual 27.
- `--adapt-swaps 96 --adapt-every 4` — small steady gain over 48/8.
- `--kv-resident 24576` tested faster but **reverted**: breaks MTP prefill on 16K prompts (`input records do not fit`) and crashed the engine once; 32768 stays.
- `STRATA_SPLIT_MTP_BATCH=1` — +1–3% prompt reads on the split, kept.
- `STRATA_PF_FUSED=1 STRATA_PF_FUSED_KQ=1` — +5% on 16K prompts, ~0 on 4K, kept. Caveat: K-quant fused prompt kernels are unmeasured upstream, so prompt numerics may differ slightly from MMQ (same class of divergence the expert cache already introduces).
- `STRATA_ROUTE_TAIL_SKIP=5`, `--vram-reserve-mib 450` (274 MiB free at load; 300 OOMs verify, 688 costs ~300 slots).

## Results (medians, ranges in files)

| Test | Prompt tok/s | Decode tok/s |
|---|---|---|
| short chat (46 tok) | — | 104.5 (98.7–122.2) |
| 4K fresh | 458.6 (458.5–458.8) | — |
| 16K fresh | 1198.2 (1194.0–1198.6) | — |
| 32K fresh (`--long`) | 1456.4 (1448.9–1459.9) | — |
| agent follow-ups (14.5K ctx) | cache reuse 68.6% | 104.1 (68.7–131.9) |

No failed requests in the suite (9 chat + 3×4K/16K/32K + 3 agent sessions, one engine session). Speed only, no correctness check — same scope as the stock suite. Rejected during tuning and **not** in this config: helper-GPU mode (`--expert-cache-device1`, decode −27% and a verify OOM on this card pair), `--expert-cache-per-layer` (hit 76–89% vs ~91%), `spec 6` (same 69% draft accept as 4), `STRATA_ADAPT_OVERLAP=0` (neutral — overlap is refused on a layer split anyway), `tail-skip 7` (slower), `pipeline-windows 0` (slower despite +214 slots).

## Files

- `bench/results/2026-10-12-community-2x-rtx-3080-ti-qwen3-8-flash-next-abliterated-q4km/README.md` — stock tool report
- `results.json` — per-request data (timings, TTFT, draft acceptance)
- `engine-log-excerpt.txt` — startup block (cache sizes, split, resident count). Note: the suite ran with `--url` against the already-running server, so the excerpt is the matching startup block from the live server log rather than tool-captured; local paths redacted, numbers untouched.