Pull requests / #2116

#2116 Community benchmark: RTX 5090 D v2 (24 GB), IQ3_S (abliterated), engine 0.1.43

open · @sobhechristl361 · 0 comentários · No GitHub

BenchmarksNVIDIA / CUDAModels & quantsDocumentationWindows

Descrição

First community report on an **RTX 5090 D v2** (the 24 GB China-market variant).

- **Hardware:** RTX 5090 D v2 24 GB (power limit 546.25 W, PCIe Gen4 x16, engine probe 28.4 GB/s host-to-device), Ryzen 5 5600X (no AVX-512, AVX-2 expert kernels), 128 GB DDR4-3600, KINGSTON SNV3S2000G 2 TB NVMe, Windows 11 26200, driver 616.56, CUDA 13, Strata / engine 0.1.43 (official Windows release).
- **Model:** ISTA-DASLab GSQ-RCO **abliterated** IQ3_S, context 262144, KV int8, expert cache auto = 5104 slots / 9.71 GiB VRAM (profile-prefilled, no eviction), MTP --spec 4 --spec-min-p 0.5, prefill auto 8192, vision on.
- **Results (medians):** chat decode 70.3 tok/s (60.7-91.1, n=9); agent-session decode 102.2 tok/s (76.6-152.8); fresh-prompt 2327 / 4572 / 4679 tok/s at 4K / 16K / 32K; TTFT 0.49 s chat, 6.89 s at 32K; agent cache hit 68.7 %.
- **Correctness:** needle_bench at 32k, depths 10/50/90: 3/3 found.
- **Limitations:** the server was live-serving the agent session that launched this benchmark (one request at a time): two short-chat TTFT outliers (19.4 s, 23.9 s) are queueing behind live requests, not engine speed; the decode and prompt throughputs are the engine's own timings and are unaffected. No VRAM/RAM peak sampling, no concurrency or sustained-thermal runs.

Measured with the built-in suite (`tools/strata_bench.py --long --url`) against the already-running server. Files: README.md + results.json + needles.json + engine-log-excerpt.txt under `bench/results/2026-10-12-community-rtx-5090-d-v2-qwen3-8-flash-next-iq3-s/`.

No site

Links install, modelos, releases.