Pull requests / #2679
#2679 bench: community report - RTX 4090 48GB (non-D), Qwen3.8-Flash-Next NVFP4-GPTQ (Maximilian228), Windows 11
open · @daifeiya · 0 commentaires · Sur GitHub
BenchmarksSetup & installNVIDIA / CUDAModels & quantsDocumentationWindowsLinux
Description
Adds a community benchmark report for an RTX 4090 48GB (non-D, Windows 11). This appears to be the first NVFP4 measurement on a 4090 in bench/results; closest 48GB reports are #2530 and #2668 (Linux, GGUF quants), and the 4090 D report #1987 (IQ3_XXS). New folder: bench/results/2026-10-16-community-rtx-4090-qwen3-8-flash-next-nvfp4/ - README.md - hardware, settings, medians - results.json - per-request data - engine-startup-lines.txt - start-up lines of the measured server (the run used an already-running server, so the tool did not write engine-log-excerpt.txt) - bench-run-output.txt - raw run log Setup: RTX 4090 48GB (49140 MiB), i9-13900KS, 127.7 GiB RAM, Windows 11 (10.0.22631), driver 617.42; engine/project 0.1.44.1; model Maximilian228/Qwen3.8-Flash-Next-NVFP4-GPTQ-Strata (NVFP4/GPTQ, GGUF pack; served as qwen3.8-flash-next-nvfp4), 524288 context, parallel 1. Run: tools/strata_bench.py --url http://127.0.0.1:8080 --long - 71 s total, medians of 3. Non-default tuning (adopted after local A/B runs; details in README): --prefill auto:32768, STRATA_PREFILL_RING=384, STRATA_PREFILL_CPU_SHARE_MAX=4096, --vram-reserve-mib 1792, elastic K/V (--kv-resident 0 --kv-grow), STRATA_GDN_CHUNKED=1. Medians: chat decode 140.7 tok/s; fresh 4K/16K/32K prompts 3697 / 5022.8 / 5240.0 tok/s; agent-session cache hit 68.9%. Notes: - On this zh-CN Windows host, tools/strata_bench.py crashed in project_version() (UnicodeDecodeError: GBK locale default vs the UTF-8 BOM of CMakeLists.txt; read_text() has no encoding argument and except OSError does not catch it). The run used PYTHONUTF8=1; suggested fix encoding="utf-8-sig". See bench-run-output.txt. - Not tested: concurrency/batch, vision, correctness, prompts beyond the suite sizes. Happy to re-run or provide more data if useful.
Sur le site
Liens install, modèles, releases.