Pull requests / #2390
#2390 bench: community results, RTX 3090 + Qwen3.8-Flash-Next UD-IQ4_XS (engine 0.1.43.1)
open · @BinbinHihi · 0 comentários · No GitHub
BenchmarksNVIDIA / CUDAModels & quantsDocumentationWindows
Descrição
## Summary Adds community benchmark results for a single RTX 3090 (24 GB) running Qwen3.8-Flash-Next in Unsloth `UD-IQ4_XS` (~4-bit) with the MTP draft head, on Windows + 128 GB RAM. Headline numbers (median of runs): - short-conversation decode: **65.5 tok/s** (range 49-81), first token ~0.40 s - agent-session decode at ~14.5K ctx: **74.8 tok/s** (60-98), prompt cache hit rate 68.8% - prompt read: 4K @ **1473 tok/s**, 16K @ **2127 tok/s** ## What changed New directory `bench/results/2026-10-13-community-rtx-3090-qwen3-8-flash-next-unsloth-ud-iq4-xs/` with: - `README.md` - full report (hardware, config, methodology, results table) - `results.json` - raw benchmark data - `engine-log-excerpt.txt` - engine log covering startup through the bench run ## Extra Notes Config used: `--gpu 0 --kv int8 --max-context 32768 --resident-budget-gib 55 --spec 4`. Expert cache ~15.5 GiB VRAM, experts mmap'd from GGUF shards in place (no experts.bin). No code changes - data only.
No site
Links install, modelos, releases.