贡献 / #2022
#2022 bench: community reports - RTX 2000 Ada 16 GB, Flash-Next IQ3_XXS at 262K, 0.1.42 and 0.1.43
open · @picmax · 0 评论 · 去 GitHub 看
BenchmarksSetup & installNVIDIA / CUDAModels & quantsDocumentationWindows
说明
## Title bench: community reports - RTX 2000 Ada 16 GB, Flash-Next IQ3_XXS at 262K, engine 0.1.42 and 0.1.43 Issue: none (community benchmark submission, per `docs/COMMUNITY_BENCHMARKS.md`) ## Summary Two reports from the same machine, model and configuration: one on 0.1.43, one on 0.1.42 as the baseline. The suite's own numbers say the two versions are level on this box. A same-machine A/B says the file-tier fix does pay here, but **only on real text** - the generated word lists used by the suite and by `tools/ab_engine.py` show nothing at all. That is the part I would value a read on. ## What changed - added `bench/results/2026-10-11-community-rtx-2000-ada-generation-qwen3-8-flash-next-iq3-xxs-0143/` - suite 1, 3 runs per test, medians with ranges - added `bench/results/2026-10-11-community-rtx-2000-ada-generation-qwen3-8-flash-next-iq3-xxs-0142/` - same, previous engine, as the A/B baseline - `COMMUNITY.md`: one row per new folder ## Extra Notes One machine, one engine loaded at a time, engine version confirmed from `BUILD.json` at every swap, 128-token greedy completions at temperature 0 with the prompt cache defeated by a unique prefix per request. **This PR's reports, 0.1.42 -> 0.1.43, same shapes the suite uses:** | | 0.1.42 | 0.1.43 | | |---|---|---|---| | short chat decode | 42.8 (36.8-51.8) | 41.2 (36.6-49.9) | ranges overlap, no separation | | agent session decode (~14.5K ctx) | 41.5 (25.5-48.8) | 42.0 (21.8-50.7) | ranges overlap, no separation | | 4K prompt | 861.3 (856.6-862.8) | 879.7 (879.4-880.0) | +2.1%, ranges do not overlap | | 16K prompt | 895.5 (893.2-899.3) | 905.0 (899.7-906.7) | +1.1% | **Prompt-shape test.** Client-side streaming rate, engines alternated in place (0.1.42, 0.1.43, 0.1.42, 0.1.43), two sessions per cell, same token count in both prompts per pair: | prompt | 0.1.42 | 0.1.43 | | |---|---|---|---| | prose, 7,159 tokens | 41.1 (40.5-41.8) | 42.5 (42.2-42.7) | +3.2% | | word list, 7,159 tokens (`ab_engine.py`'s long prompt, reproduced exactly) | 39.6 (38.0-41.3) | 39.0 (36.0-41.9) | no change | | prose, 9,975 tokens | 36.7 (34.7-38.7) | 39.5 (39.4-39.7) | +7.8% | Not a length effect: same harness, same schedule, same token count, prose gains and the word list does not. **Controls.** With the file-tier fix turned off (`STRATA_KEEP_MAPPING=1`), 0.1.43 lands back on the 0.1.42 figure on the same prose (36.4 tok/s at 10K against 36.4 for 0.1.42), so the gain tracks the fix rather than the day; a rollback control measured after the run agrees with the baseline too. Day-to-day drift on this box is 8-10%, so everything was measured inside one window. **What I could not explain.** The word list pushes *more* expert traffic to the file tier than the prose does - 5.3% of routed experts read over PCIe against 2.3%, both from the engine's own counters - yet it shows no gain. So "more file-tier traffic, more to win" is measured and wrong here, and I have no mechanism for why real text gains and a word list does not. If the 12-22 GiB Windows cases in the release notes are the intended shape and ours (about 40 GiB resident, streaming under half) is simply outside it, that would fit the numbers too. **Also checked so the result is not a reporting artifact.** 0.1.43 changed the semantics of `timings` (`prompt_n` now counts what the engine really read, the rest in `cache_n`). The engine's own decode rate and an independent client-side streaming rate agree within 0.8% on both versions, so the suite's decode column is comparable across these two builds. **Note on the reports.** The settings are this machine's tuned daily configuration (an earlier design-of-experiments study: PCIe share 0.29, KV resident 20480 MiB, prefill 4096, short-read 64, vision with a 700 MiB reserve), not a benchmark-specific setup. The fields the suite leaves blank - storage, background load, power limits - are filled in: D: NVMe holds the shards, nothing else used the GPU during the runs, and the card is at its 70 W board limit (default and maximum for this SKU).
本站相关内容
相关页面的快捷入口。