Pull requests / #2647

#2647 Community benchmark: RTX 3090, i7-12700KF, 30.4 GiB RAM, IQ2_XS at 231K (2026-10-15)

open · @Tigro14 · 0 comments · View on GitHub

BenchmarksSetup & installNVIDIA / CUDAModels & quantsDocumentation

Description

Results-only PR: adds `bench/results/2026-10-15-community-rtx-3090-qwen3-8-flash-next-iq2-xs/` (README.md, results.json, needles.json). No engine or docs changes.

**Hardware**: NVIDIA GeForce RTX 3090 24 GiB (PCIe Gen4 x16, 420 W limit, 381 W observed during the suite, max SM clock 2100 MHz); 12th Gen Intel Core i7-12700KF (12 cores, 20 threads); 30.4 GiB RAM; NVMe Fanxiang S500Pro 512 GB (xfs). Ubuntu 26.04.1 LTS, kernel 7.0.0-38-generic, driver 595.99.02, CUDA toolkit 12.9 used for the build (the driver reports CUDA 13.2). Strata project 0.1.44, commit ed4773eb, engine 0.1.43.1, local source build for sm_86. No other workloads during the run, no undervolt.

**Model and configuration**: Qwen3.8-Flash-Next IQ2_XS, shards `Qwen3.8-Flash-Next-GSQ-RCO-IQ2_XS-00001-of-00002.gguf` and `-00002-of-00002.gguf`, native-experts pack built by `tools/iq_pack.py` (sha256 in the report), `expert-profile.bin`, MTP draft `rt`, `--vision` with `mmproj-Qwen3.8-Flash-Next-BF16.gguf`; 231,072-token context, `--kv int8`, expert cache auto with resident experts, conversation cache 2048 MiB over 4 slots, prefill auto, `--spec 4 --spec-min-p 0.5`, one parallel slot. The exact launch command is in the report.

**Measured** with `./bench.sh` (`tools/strata_bench.py`, suite 1, three runs per test, medians with ranges) against the already-running server, thinking off, temperature 0: decode 112.5 tok/s (98.4-142.2) on the three short chats, prompt 2145.1 (2135.3-2147.1) tok/s at 4,019 fresh tokens and 2632.8 (2632.2-2646.9) tok/s at 16,058, TTFT 0.24 s / 1.89 s / 6.12 s; agent session cache hit 68.7 % with 3 tokens read again per session. Plus six needle checks at 32k and 128k, depths 10/50/90: 6/6 found.

**Limitations**: model loading is not timed and no engine log excerpt was written, because the server was already running; VRAM peak not measured; power draw is a single sample; the model repository and revision were not recorded by the installer; single machine, single session, no comparison with another build.

Related on this site

Quick links to related pages.