Pull requests / #2920

#2920 bench: community report, RX 6900 XT (gfx1030) on Windows, six models at 262K

open · draft · @ByteJuggler · 0 comments · View on GitHub

BenchmarksSetup & installAMD / HIPModels & quantsWindows

Description

Results only; no code changes.

**Hardware:** RX 6900 XT 16 GB (gfx1030, RDNA2, plain hipBLAS path), i7-5930K (AVX2, no AVX-512), 128 GB DDR4-2400 quad channel, Windows 11 23H2, AMD Software 26.8.1.

**Software:** engine 0.1.45, official prebuilt Windows HIP zip (ROCm 10.2.0a20260930).

**Configs tested:** six models on one config at 262,144 context: Flash-Next Q2_0, IQ3_XXS and IQ3_S, Coder IQ1_M, and Swift 1.5 IQ2_XS and IQ3_XXS. The shared flags are `--kv int8 --kv-resident 20480 --prefill 12288 --vram-reserve-mib 1024 --pool-tasks 36` with `STRATA_HIP_PROMPT_F16=1`.

**Suite:** the standard `BENCH.bat` suite, run once per model. Each model's folder has its own report, `results.json` and engine log excerpt.

| Model | Chat decode | 4K prompt | 16K prompt | Agent decode |
|---|---:|---:|---:|---:|
| Q2_0 | 50.2 | 838 | 1,014 | 62.7 |
| Swift IQ2_XS | 47.4 | 782 | 968 | 55.9 |
| Swift IQ3_XXS | 39.2 | 683 | 946 | 43.3 |
| IQ3_XXS | 39.0 | 681 | 940 | 46.4 |
| Coder IQ1_M | 38.2 | 923 | 1,126 | 46.4 |
| IQ3_S | 33.9 | 599 | 892 | 39.3 |

All figures are tok/s.

**Also measured:** two flags that setup had added when the machine had less RAM.
- Removing `--resident-experts` (Coder IQ1_M): decode +30%, 16K prompts +17%.
- Removing `--mmap-experts` (Swift IQ2_XS): decode +34%, 16K prompts +42%.

Both "before" runs are in `flag-removal/`. Setup doesn't remove these flags when RAM grows, so older installs that have since gained RAM may be slower than they need to be.

**Limitations:**
- One benchmark session per model.
- Prompts are at most 16K tokens.
- Speed only; no recall or quality check.
- The reports show a local merge commit: upstream v0.1.45 plus Python-only branches, including the BENCH fix #2919, which BENCH needs in order to start a server.

Related on this site

Quick links to related pages.