Pull requests / #2474
#2474 Community benchmark: RTX 3090, Xeon W-2235, 30.9 GiB RAM, Q2_0 at 262144 context
open · @Leowr997 · 0 Kommentare · Auf GitHub
BenchmarksNVIDIA / CUDAModels & quantsDocumentation
Beschreibung
Results-only submission per docs/COMMUNITY_BENCHMARKS.md. Adds `bench/results/2026-10-14-community-rtx-3090-qwen3-8-flash-next-q2-0-2/` (README.md, results.json, engine-log-excerpt.txt). **Hardware:** single RTX 3090 (24 GiB), Xeon W-2235 (6c/12t, no AVX-512), 30.9 GiB RAM, NVMe SSD. One DIMM, so single-channel DDR4. **Model/config:** qwen3.8-flash-next-q2_0, `--max-context 262144 --kv int8 --resident-experts --kv-resident 20480`, expert cache auto (13,547 slots / 17.44 GiB), MTP rt draft, spec 4. Strata 0.1.44.1, commit 296b59d2. **Measured (medians, 3 runs each):** chat decode 91.1 tok/s (70.2-118.1), agent-session decode 100.1 tok/s with 68.6% conversation-cache reuse, fresh-prompt ingestion 1667 / 2161 / 2199 tok/s at 4K / 16K / 32K. TTFT 0.31 s short chat, 14.6 s for a 32K fresh prompt. **Limitations:** speed only, no correctness/needle checks. KV streaming is on: engine logs show 95-98% of KV block reads hit VRAM, so the RAM-streamed remainder costs almost nothing at this context. VRAM ends up with only ~300 MiB free, which the engine warns about (#2065); no `--vram-reserve-mib` was used deliberately. No background workloads or power limits recorded.
Mehr auf der Site
Links zu Install, Modellen, Releases.