Pull requests / #2142
#2142 Community benchmark: mixed dual V100 + RTX PRO 4000 Blackwell on Windows
open · @intproduct · 0 comments · View on GitHub
BenchmarksSetup & installMulti-GPUNVIDIA / CUDAModels & quantsWindows
Description
This results-only report measures IQ2_XS with 262144 context, INT8 KV, MTP and Vision on Windows 10, EPYC 7A23 and 64 GB RAM, using the existing CUDA 12.9 engine 0.1.40. It compares the dual-V100 baseline, both three-stage GPU orders, and a dual-V100 language engine with Vision on the RTX PRO 4000 Blackwell. Each configuration has three fresh-prompt runs at 4096/32768/128000/258000 input tokens, 256 output tokens and zero prefix reuse (48 trials). With PRO first, 258000-token median TTFT changed from 158.86 to 87.24 seconds and decode from 82.2 to 95.9 tok/s. Nine long-context recall checks and API/image/tool checks passed; all four coding scores, including failed edge cases, are disclosed. The report includes sanitized configs, measured allocations, all per-run data/outputs, timing excerpts and small reproduction scripts. Limitations include different reserve settings, auto-split changes, fixed test order, background desktop activity, thermal/power limits, an older installed binary, and ambiguous coding-task boundary wording. This does not claim universal CUDA 12/Blackwell compatibility or broad Agent quality. No engine changes, binaries, model files or private transcripts are included.
Related on strata.com
Editorial links to help you install, pick models, or read release notes — not part of the upstream thread.