Pull requests / #2484
#2484 Community benchmark: 2x RTX PRO 4500 Blackwell, Swift IQ3_XXS, engine 0.1.44.1 (official suite, --long)
open · @qni-live · 0 comments · View on GitHub
BenchmarksMulti-GPUNVIDIA / CUDAModels & quantsWindowsLinux
Description
One folder under `bench/results/`, produced by `tools/strata_bench.py --long` on v0.1.44.1 (three runs per test, medians). Machine: 2x RTX PRO 4500 Blackwell (32 GB each, 200 W per card, GPU memory 14,000 MHz), Threadripper 7960X, 61 GiB RAM, Linux, driver 595.91.07, CUDA 13.2. Model: Swift IQ3_XXS (all experts resident in VRAM), layer split 25, `--pipeline-windows 2`, `--spec 5`, `--kv k8v4`, vision on, 262,144 context. | Test | Result (median) | | --- | --- | | short chat decode | 172.2 tok/s | | 4K prompt | 3,280 tok/s, first token 1.24 s | | 16K prompt | 4,732 tok/s, first token 3.41 s | | 32K prompt | 5,531 tok/s, first token 5.82 s | | agent session, follow-up turns | cache hit 68.7 %, first token 1.10 s, decode 163.7 tok/s | Notes: - This replaces the 0.1.41 entry in #1834 for the current engine; I am happy to close #1834 if you prefer a single entry. - The build carries one local patch, #1891, which does not touch anything this suite measures. - The numbers are not directly comparable with our earlier entries (they came from our own script and different prompts). - Speed only; no correctness check. - The suite stops at 32K. With our own script (same prompts as our earlier entries) the same build reads a 120,609-token prompt at 6,172 tok/s (3 runs, 6,165-6,174; 32K: 5,251), so long-prompt prefill is unchanged from 0.1.40.1 (6,148). *Written with AI assistance (Claude); numbers reviewed by me.* 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Related on this site
Quick links to related pages.