Pull requests / #2414

#2414 bench: community report, 2x RTX 5060 Ti 16 GB, Xeon E5-2690 v4, IQ3_XXS at 204,800-token context, engine 0.1.44.1

open · @R6DJO · 0 Kommentare · Auf GitHub

BenchmarksMulti-GPUNVIDIA / CUDAModels & quantsDocumentationWindowsLinux

Beschreibung

## Summary

Third community report from this machine, after [2026-10-02](https://github.com/Niko1221/Strata/tree/main/bench/results/2026-10-02-community-rtx-5060-ti-x2) (engine 0.1.35) and [2026-10-05](https://github.com/Niko1221/Strata/tree/main/bench/results/2026-10-05-community-rtx-5060-ti-x2) (engine 0.1.39). Same hardware (2x RTX 5060 Ti 16 GB, Xeon E5-2690 v4 without AVX-512, 128 GB quad-channel DDR4, Linux Mint 22.3), same model (original Flash-Next GSQ-RCO IQ3_XXS, two GPUs, fixed layer split), locally built engine **0.1.44.1** (CUDA 13.4, archs 120), repository at `296b59d2`.

Results-only change: one new folder `bench/results/2026-10-14-community-2x-rtx-5060-ti-x2/` (README, per-run JSON, needles, environment facts, engine log excerpt). No engine changes.

## What was measured

- Full fresh-prompt sweep with the [RTX 5090 serial script](https://github.com/Niko1221/Strata/blob/main/bench/results/2026-09-30-community-rtx-5090/benchmark.py): 4,096 / 32,768 / 128,000 prompt tokens, 3 runs each, greedy (temperature 0), reasoning off, 256-token output cap, unique nonce per prompt (`reused: 0` on all nine runs, all ended on the cap).
- Six needle checks (`tools/needle_bench.py`, 32K and 128K at 10/50/90 depth): **6/6 found**.

## Results (median, min-max over 3 runs)

| Prompt tokens | Prompt tok/s | Decode tok/s | TTFT s |
| ---: | --- | --- | --- |
| 4,096 | 865 (826–865) | 111.2 (92.1–116.0) | 4.76 (4.76–4.99) |
| 32,768 | 2,000 (1,994–2,037) | 101.5 (99.8–106.1) | 16.44 (16.15–16.49) |
| 128,000 | 2,776 (2,770–2,846) | 101.9 (100.8–102.1) | 46.27 (45.13–46.36) |

2026-10-05 on this machine (0.1.39): prompt 911 / 2,087 / 2,440, decode 84.8 / 86.5 / 83.8, TTFT 4.54 / 15.82 / 52.87.

## Configuration highlights (changed together since 2026-10-05)

`--max-context` 262,144 -> 204,800; `--layer-split` 25 -> 24; `--prefill` auto -> auto:16384; added `--pipeline-windows 2 --conversation-cache-slots 6 --adapt-*`; conversation cache 8,192 -> 32,768 MiB; `--vram-reserve-mib` 700 -> 900; `--pcie-frac` 0.19 -> 0.20; dropped `--remote-expert-opt`; learned expert profile rebuilt for iq3_xxs. Because the engine version and the launch configuration changed together, **no part of the deltas is attributable to a single change**; the report says so explicitly.

## Limitations

One quantization (IQ3_XXS), one synthetic workload (greedy code explanation, 256-token cap), clocks not fixed, desktop services running, RAM/VRAM not sampled during runs, single engine session. Details and per-run data in the folder's README.md.

Mehr auf der Site

Links zu Install, Modellen, Releases.