Pull requests / #2933

#2933 bench: community report, RX 7900 XTX on Windows 11, IQ2_XS and Coder IQ1_M at 262K

open · @IREvil · 0 Kommentare · Auf GitHub

BenchmarksSetup & installServer & APIAMD / HIPNVIDIA / CUDAModels & quantsDocumentationWindows

Beschreibung

Adds `bench/results/2026-10-17-community-rx-7900-xtx/`: two Qwen3.8-Flash-Next quantizations at the model's native **262,144-token context** on an **RX 7900 XTX 24 GB under Windows 11**, with the prebuilt HIP engine (0.1.44.1), on a **64 GB** machine. The [existing 7900 XTX Windows report](https://github.com/Niko1221/Strata/blob/main/bench/results/2026-10-06-community-rx-7900-xtx/README.md) covers IQ3_S with 192 GB, so this one is the same card on a typical RAM size and a different pair of quants.

## What's in it

| | Chat decode | 4K prompt | 16K prompt | 32K prompt | Agent decode | HumanEval |
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
| Flash-Next IQ2_XS | **80.2** tok/s | 1,198 tok/s | 1,367 tok/s | 1,279 tok/s | **90.2** tok/s | 81/82 |
| Coder IQ1_M | 65.6 tok/s | **1,568** tok/s | **1,631** tok/s | **1,567** tok/s | 83.4 tok/s | **82/82** |

- `iq2_xs/`, `coder-iq1_m/` — the stock `tools/strata_bench.py --long` suite, 3 runs per test, with the tool's own `results.json` and `engine-log-excerpt.txt`
- `long-context/` — a sweep past the suite's 32K ceiling out to **255K prompt tokens**, with the adapted `bench/bench_vs_llama.py` used for it included. Prefill stays within 10% of its 32K value and decode within 7-14% of its 4K value; a 255K prompt is ingested in **3.2 min (Coder) / 3.7 min (IQ2_XS)**
- `humaneval/` — an 82-problem execution-graded check per model, runner included

The two quants trade places: Coder ingests prompts 24-33% faster, IQ2_XS decodes 22% faster.

## The Windows-specific part

On a 64 GB machine the **commit limit (RAM + page file), not RAM or VRAM**, was what capped the GPU expert cache. With the 48 GB page file this machine came with, the engine dropped from its planned 11083 slots to **3667 (4.93 GiB)** — a symptom visible only in the log (`cudaMalloc ... commit left`) — and warned that it could exit silently at the ceiling (`0xC0000409`, #1607) while RAM was still free. Raising the page file to 64 GB removed the warning and the cache reached **9903 slots / 13.26 GiB**.

## Notes and limitations

- One session per model. Single decode runs vary widely; differences of a few percent between the two quants are not meaningful, the ~25-33% prefill gap is.
- The long-context sweep is a different session and prompt construction from the stock suite. Compare it within its own table, not across tables.
- No needle/recall or tool-calling test was run; HumanEval is the only correctness check here.
- Windows AMD currently can't do images (no image encoder in the Windows AMD engine), one card per model, no calibration, no `rocm-smi` telemetry.
- The dense-27B figures in the appendix are **llama.cpp**, not Strata, and are labelled as such.
- Results only: no engine, serve, setup or documentation code is touched.

Mehr auf der Site

Links zu Install, Modellen, Releases.