Pull requests / #2954
#2954 Community benchmark: 1–4× RTX 3090 scaling on EPYC 7702 — UD-IQ4_XS, 262K context
open · @bbirkinbine · 0 comments · View on GitHub
BenchmarksNVIDIA / CUDAModels & quants
Description
## Summary Adds a controlled community benchmark of 1, 2, 3, and 4 identical RTX 3090 GPUs on one EPYC 7702 host using Strata 0.1.45 and UD-IQ4_XS at 262K context. ## What changed - reports three-run medians for the stock chat, agent, 4K, 16K, and 32K long-suite workloads - includes decode throughput, prefill throughput, TTFT, ranges, and scaling - adds a separately labeled 128K fresh-prompt extension using the upstream benchmark helpers - includes raw per-request JSON, exact launch configurations, and reproducible matrix runners - records the relevant CPU, memory-channel, PCIe, GPU-board, power-limit, and software configuration ## Validation - all result and configuration JSON files parse successfully - shell runners pass bash syntax validation - the 128K companion passes Python syntax validation - every recorded benchmark result has an empty failures list - branch is based on the current upstream main and changes only the new benchmark result directory ## Extra Notes The stock suite was run with tools/strata_bench.py --runs 3 --long. The 128K extension is presented separately and measures prompt ingestion speed rather than answer quality.
Related on this site
Quick links to related pages.