贡献 / #2022

#2022 bench: community reports - RTX 2000 Ada 16 GB, Flash-Next IQ3_XXS at 262K, 0.1.42 and 0.1.43

open · @picmax · 0 评论 · 去 GitHub 看

BenchmarksSetup & installNVIDIA / CUDAModels & quantsDocumentationWindows

说明

## Title

bench: community reports - RTX 2000 Ada 16 GB, Flash-Next IQ3_XXS at 262K, engine 0.1.42 and 0.1.43

Issue: none (community benchmark submission, per `docs/COMMUNITY_BENCHMARKS.md`)

## Summary

Two reports from the same machine, model and configuration: one on 0.1.43, one on 0.1.42 as the baseline. The suite's own numbers say the two versions are level on this box. A same-machine A/B says the file-tier fix does pay here, but **only on real text** - the generated word lists used by the suite and by `tools/ab_engine.py` show nothing at all. That is the part I would value a read on.

## What changed

- added `bench/results/2026-10-11-community-rtx-2000-ada-generation-qwen3-8-flash-next-iq3-xxs-0143/` - suite 1, 3 runs per test, medians with ranges
- added `bench/results/2026-10-11-community-rtx-2000-ada-generation-qwen3-8-flash-next-iq3-xxs-0142/` - same, previous engine, as the A/B baseline
- `COMMUNITY.md`: one row per new folder

## Extra Notes

One machine, one engine loaded at a time, engine version confirmed from `BUILD.json` at every swap, 128-token greedy completions at temperature 0 with the prompt cache defeated by a unique prefix per request.

**This PR's reports, 0.1.42 -> 0.1.43, same shapes the suite uses:**

| | 0.1.42 | 0.1.43 | |
|---|---|---|---|
| short chat decode | 42.8 (36.8-51.8) | 41.2 (36.6-49.9) | ranges overlap, no separation |
| agent session decode (~14.5K ctx) | 41.5 (25.5-48.8) | 42.0 (21.8-50.7) | ranges overlap, no separation |
| 4K prompt | 861.3 (856.6-862.8) | 879.7 (879.4-880.0) | +2.1%, ranges do not overlap |
| 16K prompt | 895.5 (893.2-899.3) | 905.0 (899.7-906.7) | +1.1% |

**Prompt-shape test.** Client-side streaming rate, engines alternated in place (0.1.42, 0.1.43, 0.1.42, 0.1.43), two sessions per cell, same token count in both prompts per pair:

| prompt | 0.1.42 | 0.1.43 | |
|---|---|---|---|
| prose, 7,159 tokens | 41.1 (40.5-41.8) | 42.5 (42.2-42.7) | +3.2% |
| word list, 7,159 tokens (`ab_engine.py`'s long prompt, reproduced exactly) | 39.6 (38.0-41.3) | 39.0 (36.0-41.9) | no change |
| prose, 9,975 tokens | 36.7 (34.7-38.7) | 39.5 (39.4-39.7) | +7.8% |

Not a length effect: same harness, same schedule, same token count, prose gains and the word list does not.

**Controls.** With the file-tier fix turned off (`STRATA_KEEP_MAPPING=1`), 0.1.43 lands back on the 0.1.42 figure on the same prose (36.4 tok/s at 10K against 36.4 for 0.1.42), so the gain tracks the fix rather than the day; a rollback control measured after the run agrees with the baseline too. Day-to-day drift on this box is 8-10%, so everything was measured inside one window.

**What I could not explain.** The word list pushes *more* expert traffic to the file tier than the prose does - 5.3% of routed experts read over PCIe against 2.3%, both from the engine's own counters - yet it shows no gain. So "more file-tier traffic, more to win" is measured and wrong here, and I have no mechanism for why real text gains and a word list does not. If the 12-22 GiB Windows cases in the release notes are the intended shape and ours (about 40 GiB resident, streaming under half) is simply outside it, that would fit the numbers too.

**Also checked so the result is not a reporting artifact.** 0.1.43 changed the semantics of `timings` (`prompt_n` now counts what the engine really read, the rest in `cache_n`). The engine's own decode rate and an independent client-side streaming rate agree within 0.8% on both versions, so the suite's decode column is comparable across these two builds.

**Note on the reports.** The settings are this machine's tuned daily configuration (an earlier design-of-experiments study: PCIe share 0.29, KV resident 20480 MiB, prefill 4096, short-read 64, vision with a 700 MiB reserve), not a benchmark-specific setup. The fields the suite leaves blank - storage, background load, power limits - are filled in: D: NVMe holds the shards, nothing else used the GPU during the runs, and the card is at its 70 W board limit (default and maximum for this SKU).

本站相关内容

相关页面的快捷入口。