#2262 Community benchmark: RTX 5080 + RTX 5060 Ti 16 GB hold layers at 262K on Unsloth UD-Q4_K_XL — engine 0.1.43, two launch configs (fork-free submission, files attached)

说明

## Title
Community benchmark: RTX 5080 + RTX 5060 Ti 16 GB, Unsloth UD-Q4_K_XL at 262K — two launch configs on engine 0.1.43
Issue: none (results-only contribution)

## Summary
Full reports in `bench/results/2026-10-12-community-rtx-5080-5060-ti-qwen3-8-flash-next-unsloth-ud-q4-k-xl/` and `…-ud-q4-k-xl-2/` (`README.md`, `results.json`, `engine-log-excerpt.txt` each). A heterogeneous 16 GB + 16 GB pair — different silicon, equal VRAM, both cards hold layers — runs Unsloth Qwen3.8-Flash-Next UD-Q4_K_XL at 262,144 context: 71.7 tok/s short-chat decode on the auto-split config, 57.4 on the tuned fixed split, ~68.7 % conversation cache hit under agent-style resends. Two passes, same day, same build, different launch configs.

## Folders to add under `bench/results/`
- `bench/results/2026-10-12-community-rtx-5080-5060-ti-qwen3-8-flash-next-unsloth-ud-q4-k-xl/` — pass 1 (tuned split + control vector)
- `bench/results/…-ud-q4-k-xl-2/` — pass 2 (auto split + adaptive)
Nothing else. No engine code, no defaults, no headline claims, no index rows — filed fork-free via a GitHub issue exactly as the guide allows ("open a pull request that adds the folder … **or paste README.md into a GitHub issue**"); the landed precedent `c50a217a` added the folder only and maintainers batch the index rows.

## Hardware
- GPUs: RTX 5080 16 GB (GB203) + RTX 5060 Ti 16 GB (GB206), driver 615.78.08; CUDA version not read. 32 GiB total under `--layer-split` (both cards hold layers; neither is a cache-only helper).
- CPU / board: Ryzen 7 7800X3D (8C/16T), ASUS ROG CROSSHAIR X670E GENE, 93.9 GiB DDR5.
- Link widths: 5080 Gen5 x16 (at max, sysfs); 5060 Ti read back Gen5 x4 at idle after the runs (device is x16-capable) — in-run width not recorded, so the x4 reading may be idle downtraining.
- OS: Linux 7.2.8-5-omarchy-bore (Arch-based, Omarchy); Strata project 0.1.43, engine commit `7695859a` (tagged ancestor of v0.1.43), source build (CMake build tree of that checkout).

## Configuration
Both passes: `--max-context 262144 --kv int8 --kv-resident 32768 --resident-budget-gib 66 --spec 4 --mtp rt --spec-min-p 0.5 --vision --expert-cache auto --expert-profile expert-profile.bin`.
- Pass 1 (tuned): `--layer-split 43 --split-device 1` + speed-projection control vector (`…speed-projection.gguf:1.0`, layers 4-44) `--pcie-frac 0.20 --pipeline-windows 2 --pool-workers 4`, `STRATA_STAGE_TRIM=1`.
- Pass 2 (stock-ish): `--layer-split auto --ple-io direct` + adaptive tier `1/160/0.97`, no control vector, no stage trim.
Full command blocks are in each README, byte-identical to what `tools/strata_bench.py` recorded.
Both custom files ship in the repo and are byte-identical to the blobs at `7695859a` (sha256s in the READMEs): `data/expert-profile.bin`, and for pass 1 only `data/experimental-speed-projection/` (tracked since `6abee7f3`, experimental, off by default; its sidecar records the derivation tooling and base shard — a UD-Q2_K_XL shard, a different quant from this run). A fresh clone reproduces both flags as-is.

## Results
`tools/strata_bench.py` suite 1, medians (range), 3 runs/test, unique-tag prompts (no cache reuse in fresh-prompt tests):

| Test | Pass 1 (tuned split) | Pass 2 (auto split) |
| --- | --- | --- |
| short chat decode tok/s | 57.4 (48.0-72.5) | 71.7 (56.2-90.3) |
| short chat TTFT s | 0.51 (0.39-0.64) | 0.37 (0.32-0.45) |
| 4K fresh prompt tok/s (TTFT s) | 1,883.8 (1,523.2-1,884.6) (2.15) | 1,879.7 (1,567.8-1,893.8) (2.16) |
| 16K fresh prompt tok/s (TTFT s) | 2,899.6 (2,701.6-2,903.4) (5.56) | 2,905.2 (2,676.3-2,911.7) (5.55) |
| agent-turn decode tok/s | 63.2 (48.7-86.2) | 74.0 (44.5-95.2) |
| agent-turn cache hit | 68.6 % (68.5-68.7) | 68.7 % (68.7-68.8) |

## What this adds over existing entries
- The first heterogeneous 16 GB + 16 GB pair in the index: neighbors are identical-card pairs (#2008: 2x 5060 Ti 16 GB, IQ2_XS, still open), a larger heterogeneous pair at a bigger budget (#2012: RTX 5090 32 GB + RTX 3090 24 GB, IQ3_S, 131K), or big-card-plus-small-helper splits (#1681: 5080 + 8 GB 4060 Ti, helper caches overflow only). Both cards carrying real layers at exactly 16+16 GB is the upgrade path most 16 GB owners would actually buy.
- Highest quality tier filed on a 32 GB total at 262K: the same-total neighbor #2008 files IQ2_XS.
- Same engine and 32 GB total as #2008, so decode bands read across: their identical 5060 Ti pair on IQ2_XS decodes 85.3; this pair carries a higher quality tier and decodes 57.4-71.7.
- Two launch configs on one build in one submission: auto/adaptive beats the fixed split + control vector on this pair (57.4 → 71.7 short chat, TTFT 0.51 → 0.37), while fresh-prompt prefill is identical within noise (1,883.8 vs 1,879.7 at 4K), localizing the difference to the decode path rather than the prefill/PCIe path.

## Validation (how to check these numbers)
- Every median and range recomputes from per-request arrays in each folder's `results.json`; both runs completed with zero failures and nothing skipped.
- Launch commands in the READMEs match the harness-recorded `machine.launch` byte-for-byte; engine build `7695859a` is the tagged ancestor of v0.1.43.
- Ruled out / not claimed: the 57.4 → 71.7 short-chat gap is **not** attributed. The configs differ (split, control vector, stage trim) and background load was **not recorded**; neither explanation is separable from this data. The prefill-parity observation is a localization, not a cause.
- Not tested: correctness/quality, needle recall, concurrency (`--clients`), prompts beyond 16K, vision requests. Not recorded: background load, power draw, in-run PCIe link width, CUDA version, storage details.

## Extra Notes
Prompt rates at 4K/16K sit mid-band vs #2008 despite the stronger CPU — consistent with the second card behind fewer lanes, but lane count during the runs was not recorded, so that stays a hypothesis the next 2x16 GB submission can test.


## File manifest (sha256, sizes) — folder contents verbatim in the linked gist

| file | sha256 | bytes |
|---|---|---:|
| `2026-10-12-community-rtx-5080-5060-ti-qwen3-8-flash-next-unsloth-ud-q4-k-xl/README.md` | `6c6bb0fa044af2600c77a8ac95a814f2a281fba555f391e843da1a6aa1f325cd` | 6185 |
| `2026-10-12-community-rtx-5080-5060-ti-qwen3-8-flash-next-unsloth-ud-q4-k-xl/engine-log-excerpt.txt` | `8ed2c30b3bae8b47ac035c6f1d40281def420c4f2153a5779aa4a0f2d9030f24` | 508 |
| `2026-10-12-community-rtx-5080-5060-ti-qwen3-8-flash-next-unsloth-ud-q4-k-xl/results.json` | `8e4d250aa58fd57d1e755e54a9761578af5e86034aa00c4c995f06e36373f2f3` | 21546 |
| `2026-10-12-community-rtx-5080-5060-ti-qwen3-8-flash-next-unsloth-ud-q4-k-xl-2/README.md` | `dda20d48f768b361242626c437c10ceedb3c8bcb3020a814db3da584b5d82e86` | 5518 |
| `2026-10-12-community-rtx-5080-5060-ti-qwen3-8-flash-next-unsloth-ud-q4-k-xl-2/engine-log-excerpt.txt` | `a8d7da7f1d288213e659a93053e8af5a00b8bdd3cd4d40f304a2c8696f16f11b` | 510 |
| `2026-10-12-community-rtx-5080-5060-ti-qwen3-8-flash-next-unsloth-ud-q4-k-xl-2/results.json` | `39b7579825eafe2c65e47d8ff3dec3a45f8deaf1aa6d826b8dafe30ee47859c5` | 21226 |

---

## Full README files (pasted per the guide)

### `2026-10-12-community-rtx-5080-5060-ti-qwen3-8-flash-next-unsloth-ud-q4-k-xl/README.md`

```markdown
# Community benchmark on NVIDIA GeForce RTX 5080 + NVIDIA GeForce RTX 5060 Ti

Measured on 2026-10-12 by beplee with `tools/strata_bench.py` (suite 1, 3 runs per test, medians with ranges). Speeds are the engine's own timings; first-token times are measured by the client over HTTP. Speed only: no correctness check was run.

## Hardware and software

- GPU and VRAM: NVIDIA GeForce RTX 5080 + NVIDIA GeForce RTX 5060 Ti, 32 GiB VRAM; PCIe link: 5080 read Gen5 x16 (at its max), 5060 Ti read back Gen5 x4 after the runs at idle (device is x16-capable) — width during the runs was not recorded
- CPU: AMD Ryzen 7 7800X3D 8-Core Processor (8 cores, 16 threads); installed RAM: 93.9 GiB; storage: not measured
- OS: Linux 7.2.8-5-omarchy-bore; GPU driver: 615.78.08; CUDA: not read
- Strata: project 0.1.43, commit 7695859a; engine 0.1.43; source build (CMake build tree of that checkout)
- Background workloads and power limits: not recorded

## Model and configuration

- Model and quantization: qwen3.8-flash-next-unsloth-ud-q4_k_xl
- Vision encoder; custom packs or profiles: vision on via `--vision` (engine's built-in encoder; encoder file not recorded). `expert-profile.bin` is the repo-bundled profile (`data/expert-profile.bin`, byte-identical to the repo blob at `7695859a`; 196,632 B, sha256 8f59b4aa8873209dff11c11e37bcda9529a1335b724a1afeea37bf6388975baf) — a fresh clone reproduces it. This pass also loads the repo-tracked experimental control vector `data/experimental-speed-projection/Qwen3.8-Flash-Next-experimental-speed-projection.gguf` (483,520 B, sha256 ef0724c5b79297e481017be85769832bc26eb220c984ecba2b0e1b0c8426312b, byte-identical to the repo blob; tracked since `6abee7f3`, marked experimental, off by default). Its sidecar JSON records the derivation tooling and the base-model shard (a UD-Q2_K_XL shard — a different quant from this run's UD-Q4_K_XL) plus KL metrics. The second pass runs without it.
- Context: 262144 tokens; parallel slots: 1
- Settings: the install's own configuration, unchanged from that moment's install state; thinking off for the suite; temperature 0. Note: a second same-day pass (`2026-10-12-community-rtx-5080-5060-ti-qwen3-8-flash-next-unsloth-ud-q4-k-xl-2/`) used a different launch config (auto layer split with adaptive swaps, no control vector, no `STRATA_STAGE_TRIM`), so differences between the two reports are not attributed to a single cause.

```text
strata --pack unsloth-ud-q4_k_xl --native Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf --expert-profile expert-profile.bin --expert-cache auto --prefill auto --spec 4 --mtp rt --max-context 262144 --kv int8 --kv-resident 32768 --resident-budget-gib 66 --vision --vram-reserve-mib 700 --layer-split 43 --split-device 1 --control-vector-scaled Qwen3.8-Flash-Next-experimental-speed-projection.gguf:1.0 --control-vector-layer-range 4 44 --cvec-mode project --cvec-dir per-layer --pcie-frac 0.20 --pipeline-windows 2 --spec-min-p 0.5 --adapt-async 1 --pool-workers 4
```

Environment of the config: `STRATA_ROUTE_RESIDENT=0.25`, `STRATA_ROUTE_TAIL_SKIP=7`, `STRATA_ADAPT_OVERLAP=1`, `STRATA_STAGE_TRIM=1`

## Method

Run by `python tools/strata_bench.py` against the HTTP API of a Strata server (started by the benchmark from the install configuration); model loading is not timed. One warm-up chat and a 1K prompt are discarded first. Every prompt starts with a unique tag, so the conversation cache cannot help the fresh-prompt tests.

- short chat: the three questions of `tools/ab_engine.py`, up to 100 new tokens, 3 runs
- 4K / 16K prompts: generated word lists sized by a calibration request, 24 new tokens, 3 fresh prompts each
- agent session: system prompt and four tools, then 5 turns that each add about 1500 words of a project file and continue from the model's own earlier answers (the way agent clients resend a conversation), 64 new tokens per turn, 3 sessions; the cache numbers come from the answer's usage (`cached_tokens`) and timings (`prompt_n`)
- concurrency (when run): N requests sent at the same moment, total tokens over the wall time of the burst, latency over all requests

## Results

| Configuration | Actual prompt tokens | Reused tokens | Generated tokens | Runs | Prompt tok/s median and range | Decode tok/s median and range | TTFT seconds median and range |
| --- | ---: | ---: | ---: | ---: | --- | --- | --- |
| short chat | 46 (43-51) | 0 | 100 (100-100) | 9 | not measured | 57.4 (48.0-72.5) | 0.51 (0.39-0.64) |
| 4K prompt, fresh | 4019 (4019-4019) | 0 (0-0) | 24 cap | 3 | 1883.8 (1523.2-1884.6) | not measured | 2.15 (2.15-2.66) |
| 16K prompt, fresh | 16058 (16058-16058) | 0 (0-0) | 24 cap | 3 | 2899.6 (2701.6-2903.4) | not measured | 5.56 (5.55-5.97) |
| agent session, follow-up turns | 14442 (14420-14453) a