Issues / #3659
#3659 [Engine] nv_defaults for sm_89 (Ada): the sm_86 switch set pays on an RTX 4000 SFF Ada 20 GB - scope agreement before a default change
open · @ctr-137 · 0 comentários · No GitHub
BenchmarksNVIDIA / CUDAModels & quantsDocumentation
Descrição
## Problem `src/core/nv_defaults.cpp` gives its bitwise decode switch set to sm_86 and sm_60 cards only. An sm_89 (Ada) card gets the empty table — the code path at `mj == 8 && mn == 6` simply does not match it. On our RTX 4000 SFF Ada 20 GB (70 W cap, Hetzner host, i5-13500) we measured that the **full sm_86 set as a whole pays on sm_89**, and we would like maintainer agreement to add an sm_89 branch before opening the PR (CONTRIBUTING: a change to a default is agreed in an issue first). ## Measured (same hardware and workload, every changed setting named) Five `strata bench run` standard-v1 draft.1 runs on one machine on 2026-10-21, fresh engine per run, same build (0.1.46.2), same serving flags (`--spec 4 --spec-min-p 0.5 --mtp --max-context 262144 --kv int8 --kv-resident 32768 --expert-cache auto --prefill auto --vision cpu --pool-workers 9`), model Qwen3.8-Flash-Next GSQ-RCO IQ3_XXS: - plain (no switches): decode medians 61.95 / 52.76 / 48.80 tok/s (short / 4K / 16K) - the nine sm_86 switches exported in the start script, twice: 63.65 / 61.18 / 53.93 and 65.57 / 56.07 / 52.46 → **+4.3% / +11.1% / +9.0%** against the plain run - the same set applied **by the engine itself** (a patched `nv_defaults.cpp` with an added `mj == 8 && mn == 9` branch; nothing exported): 66.33 / 60.81 / 55.46 — the gain reproduces, so the default path works, not just the environment trick - all five runs `complete`, **12/12 sanity** each; prompt rates unchanged Raw runs: the folders `bench/results/2026-10-21-standard-rtx-4000-sff-ada-generation-*` proposed in the benchmark report PR (linked below when opened). ## Proposed behavior - An `sm_89` branch in `nv_default_env` carrying the nine sm_86 switches unchanged, with `STRATA_SM89_PERF=0` as its off switch (mirroring the sm_86/sm_60 gates; a switch the user set is kept as before). - The startup line currently prints "sm_86 (Ampere) ... STRATA_SM86_PERF" for *any* major-8 card (`nv_defaults.cpp` checks `cc_major_of(major) == 8`); with an sm_89 table it would misname the architecture and name the wrong gate. The change also makes the line print the table it actually applied. - `docs/DETAILS.md` and `include/strata/core/nv_defaults.hpp` updated with the sm_89 row and these numbers. - A prepared branch exists (`ctr-137/Strata` `engine-nv-defaults-sm89`, two commits: code + docs) — the PR follows on your go. ## Tradeoffs, honestly - **One card measured.** It is a 70 W-capped Ada pro card; a 4070/4080/4090 or an L4 may land differently, and the set was measured **as a whole** here, not switch by switch (the sm_86 table was built drop-one on two cards). If the maintainers want drop-one runs on sm_89 before defaulting, we can run them on this box. - Quality risk is structurally zero: every switch in this set is bitwise by construction (the identity gate that made them defaults), and the sanity suites passed 12/12 in all five runs. - Cards whose visible GPUs are not uniformly sm_89 keep the current behavior (the table is only applied when every visible CUDA device shares the capability — unchanged). Related: the benchmark report (same hardware, five protocol runs) and #3453 / #3452 (an RTX 4070's IL tables and benchmark, a second sm_89 data point from a different contributor).
No site
Links install, modelos, releases.