Pull requests / #2739
#2739 bench: mixed RTX 3090, RTX 3060 and dual V100 Q4_K_M community report
open · @techrian · 0 Kommentare · Auf GitHub
BenchmarksMulti-GPUAMD / HIPNVIDIA / CUDAModels & quantsDocumentationLinux
Beschreibung
## Summary Add a community report for **RTX 3090 24 GiB + RTX 3060 12 GiB + two V100 32 GiB**, Ryzen 7 9700X, 64 GB class RAM, Linux/CUDA 12.8. The measured deployment is unmodified **Strata 0.1.43.1**, commit `a0116b2782a5b82622358c04a06496485d12b2d6`, with OrcaRouter Qwen3.8-Flash-Next-Uncensored Q4_K_M, 262144 context, FP16 KV, prefill 6144, RAM PLE, native MTP and resident vision. ## What changed - Add a dated report with physical/engine GPU mapping, observed PCIe widths, model/build/configuration details and limitations. - Include all 22 unique earlier throughput observations and a summary CSV; the final 6144/default-sampling/swap-blocked group has three runs and reports median/range. - Include 72 per-episode records from a custom native-tool assistant workload: 24 tasks repeated three times, 69 strict complete successes. - Link the complete configurations, task fixtures, raw native requests/responses, artifacts, grading audit and 14-sheet workbook in the pinned [mixed-gpu-llm-lab snapshot](https://github.com/techrian/mixed-gpu-llm-lab/tree/7b2f013786a478a59417ebe1937b477eb42dcc08). - Add the report to `docs/COMMUNITY_BENCHMARKS.md`. ## Measurement scope The throughput measurements are October 14–15; the agent episodes are October 16. All earlier conditions and failures are retained. Earlier groups differ in sampling, reserve and swap policy and are not presented as a single controlled A/B. The custom agent result is not an official BFCL/tau/Terminal-Bench/Hermes score. The actual Hermes application was not invoked. The report discloses the scope-ambiguous quote task, equivalent-representation grading audit, all original verdicts, and the empty-final-response case. The separately linked GLM comparison uses Project Maya 1.1.11 / Maya-M. It compares the two deployed stacks, with different native quantization/KV/MTP settings, rather than isolating engine or base-model quality. ## Validation and limits - Numerical episode metrics and aggregate statistics match the original completed benchmark; all 238 task fixture/artifact files are byte-preserved in the full lab archive. - The public workbook was regenerated from exported records and checked for 144 run rows, 735 tool calls, cached formulas, chart presence and XLSX integrity. - Source and configuration provenance, prompts, budgets, repetitions, warm-up and timing boundaries are documented. - No inference was rerun on current upstream `main` for this submission; this reports the pinned working build. No claim about a newer build or a universal optimum is made. - No runtime code, defaults or backend build files change. No CUDA/HIP/SYCL build was needed for this results-only contribution. I searched existing reports and open PRs; there are other V100 combinations, but not this four-card RTX3090/RTX3060/dual-V100 deployment and paired assistant workload.
Mehr auf der Site
Links zu Install, Modellen, Releases.