Pull requests / #234
#234 Bench: RTX 3090 / dual RTX 3090 on EPYC 7453
closed · @mad9home · 0 comentários · No GitHub
BenchmarksMulti-GPUAMD / HIPNVIDIA / CUDAModels & quants
Descrição
## Summary Add a preliminary native benchmark report for an EPYC 7453 KVM guest with 2x RTX 3090s. It includes the IQ3_XXS single-GPU matrix from 1K through 262K, native Coder/IQ3_XXS/IQ3_S dual-GPU 32K results, the Coder fixed-K sweep, correctness checks, scrubbed guest hardware facts, and raw per-run JSON. Related issue: https://github.com/Niko1221/Strata/issues/165 ## Method Measured with native Strata v0.1.26, locally built from ac8b251, temperature 0, reasoning effort none, and 256 requested output tokens. Three runs per speed cell where noted; medians and actual prompt-token counts are reported. The local API harness uses generated tasks from the public Strata source tree because the upstream code-agent prompts and harness were not available when these runs were collected. These results are preliminary and are not direct comparisons to the published code-agent prompt results. ## Measured on Ubuntu 24.04.5 KVM guest; AMD EPYC 7453, 20 vCPU, AVX2; 165 GiB guest-visible RAM; 2x RTX 3090 24 GB; NVIDIA driver 595.91.07. Both GPUs were configured to a 280 W power limit. Physical DIMM topology and memory bandwidth are not verified. ## Validation - IQ3_XXS single-GPU speed cells: 1K / 4K / 32K / 128K / 262K, three runs each. - Native Coder IQ1_M: 32K single-GPU, dual-GPU auto, and fixed K=18/22/24/26/28/30. K=18 has one successful run; a repeat startup OOMed during a dense-weight upload and is not treated as a three-run median. - IQ3_XXS and IQ3_S dual-GPU auto at 32K. - IQ3_XXS and IQ3_S needle checks passed 3/3 at 8K / 16K / 28K; both 16K two-turn checks reused the checkpoint. - Native service health/API smoke test passed; llama-swap model containers were stopped during benchmarking. ## Known limits No calibrated/default comparison, physical memory-bandwidth measurement, STRATA_SPLIT_TIMING stage timing, or Coder dual-GPU correctness checks yet. Fixed-K prompts were independent local-corpus prompts, so small differences between K values need a paired rerun with the upstream prompts. No Docker measurements are included. hw.txt records verified guest-visible facts only.
No site
Links install, modelos, releases.