#3290 [Benchmark] 3x GPU (RTX 5060 Ti + 2080 Ti + 3080 Ti): 0.2.0-beta, Qwen3.8-Flash-Next IQ3_XXS at 262K

本文

## Problem and result

A community benchmark report measuring Qwen3.8-Flash-Next (125B MoE, IQ3_XXS, ~3.4 bpw) running across a heterogeneous 3-GPU desktop setup using Strata 0.2.0-beta (`project 0.1.45.3`, engine `0.1.45`, commit `e88ee39d`) with MTP speculative decoding (`--spec 4`, `--spec-min-p 0.65`) at native context 262,144 tokens.

The pull request adds `bench/results/2026-10-19-community-rtx-5060-ti-plus-rtx-2080-ti-plus-rtx-3080-ti-qwen3-8-flash-next-iq3-xxs/`: the tool-written `README.md` and the per-request `results.json` (`schema: 1`).

### Hardware and Software
- **GPUs:** 3 NVIDIA GPUs (~39.8 GiB total VRAM)
  - Stage 0: NVIDIA GeForce RTX 5060 Ti 16 GiB (`sm_120`, Blackwell)
  - Stage 1: NVIDIA GeForce RTX 2080 Ti 11 GiB (`sm_75`, Turing)
  - Stage 2: NVIDIA GeForce RTX 3080 Ti 12 GiB (`sm_86`, Ampere + MTP head)
- **Host CPU:** Dual Intel Xeon E5-2683 v4 (32 cores / 64 threads, NUMA interleave)
- **RAM:** 184.5 GiB DDR4
- **OS / Drivers:** Linux 7.2.6-generic, NVIDIA Driver 615.71.09, CUDA 13.4
- **Engine / Branch:** Strata 0.2.0-beta (`origin/main` commit `e88ee39d`), native unified CUDA build (`sm_75`, `sm_86`, `sm_120`)

### Configuration
- **Model:** `Qwen3.8-Flash-Next-GSQ-RCO-abliterated-IQ3_XXS`
- **Context:** 262,144 tokens (`--kv k8v4`)
- **Parallel Pipeline:** `--layer-split 17,29` across GPUs `[1, 0, 2]`
- **Speculative Decoding:** `--spec 4 --spec-min-p 0.65 --mtp rt`
- **Vision:** `--vision` sidecar enabled

### Results Summary (3 runs each, medians with ranges)
- **Short Chat Decode:** 81.1 tok/s median (76.0 – 102.5 tok/s, n=9), TTFT 0.44 s (0.41 – 0.47 s)
- **4K Prompt Prefill:** 1,035.1 tok/s median (1,025.0 – 1,047.7 tok/s), TTFT 3.91 s
- **16K Prompt Prefill:** 2,052.6 tok/s median (2,042.6 – 2,059.9 tok/s), TTFT 7.86 s
- **32K Prompt Prefill:** 2,814.2 tok/s median (2,797.2 – 2,815.6 tok/s), TTFT 11.44 s
- **Agent Session:** 99.3 tok/s median decode (79.1 – 125.0 tok/s), 69.3% cache hit rate, first-turn TTFT 1.95 s, turn TTFT 3.47 s across 6 turns.

## Approach

Results only: one folder `bench/results/2026-10-19-community-rtx-5060-ti-plus-rtx-2080-ti-plus-rtx-3080-ti-qwen3-8-flash-next-iq3-xxs/` containing the tool-generated `README.md` and `results.json`, per `docs/COMMUNITY_BENCHMARKS.md` "Submit a report" guidelines. No engine or documentation modifications.

## Validation

Passed:
- Suite executed via `python tools/strata_bench.py --url http://127.0.0.1:8080 --config strata-sc117-abliterated.json --long` on `main` at `e88ee39d` (0.2.0-beta).
- All requests completed with 0 errors / 0 failures across short chat, 4K/16K/32K prompts, and 6-turn agent sessions.
- `results.json` validates with `schema: 1`.
- Clean branch rebased on `origin/main` `e88ee39d`.

Failed:
- None observed.

Not run:
- No correctness check (speed-only suite as described in `docs/COMMUNITY_BENCHMARKS.md`).
- Multi-client concurrency test (`--clients N`).

## Limits

Speed suite only; background loads during the run were not recorded.