Pull requests / #2954

#2954 Community benchmark: 1–4× RTX 3090 scaling on EPYC 7702 — UD-IQ4_XS, 262K context

open · @bbirkinbine · 0 commentaires · Sur GitHub

BenchmarksNVIDIA / CUDAModels & quants

Description

## Summary

Adds a controlled community benchmark of 1, 2, 3, and 4 identical RTX 3090 GPUs on one EPYC 7702 host using Strata 0.1.45 and UD-IQ4_XS at 262K context.

## What changed

- reports three-run medians for the stock chat, agent, 4K, 16K, and 32K long-suite workloads
- includes decode throughput, prefill throughput, TTFT, ranges, and scaling
- adds a separately labeled 128K fresh-prompt extension using the upstream benchmark helpers
- includes raw per-request JSON, exact launch configurations, and reproducible matrix runners
- records the relevant CPU, memory-channel, PCIe, GPU-board, power-limit, and software configuration

## Validation

- all result and configuration JSON files parse successfully
- shell runners pass bash syntax validation
- the 128K companion passes Python syntax validation
- every recorded benchmark result has an empty failures list
- branch is based on the current upstream main and changes only the new benchmark result directory

## Extra Notes

The stock suite was run with tools/strata_bench.py --runs 3 --long. The 128K extension is presented separately and measures prompt ingestion speed rather than answer quality.

Sur le site

Liens install, modèles, releases.