Pull requests / #2379
#2379 bench: community reports, 1x/2x/6x Tesla V100-PCIE-32GB scaling, Unsl…
open · @zwcf5200 · 0 commentaires · Sur GitHub
BenchmarksSetup & installMulti-GPUNVIDIA / CUDAModels & quantsSecurityDocumentationLinux
Description
Results-only PR: four report folders in bench/results/ (README + per-run results.json each), four COMMUNITY.md rows and one docs/COMMUNITY_BENCHMARKS.md entry. No engine changes. Hardware: 1x / 2x / 6x Tesla V100-PCIE-32GB (sm_70, PCIe, auto layer split on the multi-card arms), 2x Xeon Silver 4116, 502.6 GiB RAM, Linux, driver 580.178.04, CUDA 13.0. Engine 0.1.43.1 (a0116b2) as installed, Unsloth UD-Q4_K_XL with its expert profile, --spec 4 --mtp rt --kv int8 --resident-budget-gib 71. Results (tools/strata_bench.py --long, 3 runs per cell, medians): - Fresh-prompt prefill at 4K/16K: 596 / 1,017 (one card), 992 / 1,646 (two), 1,281 / 2,422 tok/s (six, 32K context). Short-chat decode: 37.5 / 63.3 / 70.6 tok/s across the ladder. - The same six cards at 262,144 context take a 32K prompt at 3,388 tok/s and decode 80.1 tok/s. - Agent sessions reuse 68.5-68.8% of the follow-up turns' tokens (5-turn sessions, ~14.5K tokens). To my knowledge this is the first six-V100 measurement on record (previous largest: 4x, #2048). Complements #1508 (2x PCIe V100, IQ4_XS, older engine) and issue #2260 (4x PCIe 32GB, same engine). Limits: speed only (no needle or correctness runs), one machine, power limits and background load not recorded, 32K context skipped on the 32,768-token arms. Prepared with an AI assistant (Zed agent) from the runs on the machine above; the numbers are the author's own measurements.
Sur le site
Liens install, modèles, releases.