Pull requests / #2066
#2066 bench: dual Quadro RTX 6000 results, upgrade A/B and NUMA comparison
open · @ambu50 · 0 コメント · GitHub で見る
BenchmarksMulti-GPUNVIDIA / CUDAModels & quantsDocumentation
本文
## Summary Adds a results-only community report for a dual Quadro RTX 6000 24 GiB / dual Xeon Gold 6138 ThinkStation P720 running Unsloth UD-Q4_K_XL. - Official `tools/strata_bench.py` suite on v0.1.43, three repetitions and `--long`. - Standard-suite medians: chat decode 65.7 tok/s, agent follow-up decode 70.5 tok/s; fresh 4K/16K/32K prefill 840.8/1542.0/1796.7 tok/s. - Supplementary old/new/new/old comparison: median short decode 66.95 -> 74.80 tok/s (+11.7%) from 0.1.40.3 to 0.1.43; prefill essentially unchanged. - Supplementary NUMA experiment on 0.1.40.3 after expanding GPU-local socket RAM: interleave outperformed node-0-only memory placement on the tested corpus. - Raw per-request records, timing excerpts, verified placement summaries, sanitized config, custom corpus and reproduction instructions included. ## Scope and limitations The standard suite and custom comparisons are reported separately. Two launches per custom configuration/version; three repeats in the standard suite. Configured context is 262144, largest tested input is 43440 tokens. No long-context recall or quality evaluation. The version comparison uses release defaults, including STRATA_ROUTE_TAIL_SKIP=7, which can change answers. No claim of byte-identical output or causal attribution of speedups. CPU vision helper was retained. The official report's auto-detected CUDA 13.2 field is explicitly identified as driver capability; the engine was built with CUDA toolkit 12.4 for sm_75. No engine changes or model files are included. Adds a report link to docs/COMMUNITY_BENCHMARKS.md. Related measurement context: #1900 (opt-in NUMA placement proposal). This report provides workload-specific measurements; it does not implement that proposal or claim that GPU-local placement always helps. ## Validation - Official suite completed; Hermes restored and Strata healthy. - All JSON artifacts parse. - Personal paths and unrelated live-user traffic excluded from packaged data. - `git diff --cached --check` passes.
関連リンク
インストール・モデル・リリースへの站内リンク。