Pull requests / #2920
#2920 bench: community report, RX 6900 XT (gfx1030) on Windows, six models at 262K
open · draft · @ByteJuggler · 0 comentários · No GitHub
BenchmarksSetup & installAMD / HIPModels & quantsWindows
Descrição
Results only; no code changes. **Hardware:** RX 6900 XT 16 GB (gfx1030, RDNA2, plain hipBLAS path), i7-5930K (AVX2, no AVX-512), 128 GB DDR4-2400 quad channel, Windows 11 23H2, AMD Software 26.8.1. **Software:** engine 0.1.45, official prebuilt Windows HIP zip (ROCm 10.2.0a20260930). **Configs tested:** six models on one config at 262,144 context: Flash-Next Q2_0, IQ3_XXS and IQ3_S, Coder IQ1_M, and Swift 1.5 IQ2_XS and IQ3_XXS. The shared flags are `--kv int8 --kv-resident 20480 --prefill 12288 --vram-reserve-mib 1024 --pool-tasks 36` with `STRATA_HIP_PROMPT_F16=1`. **Suite:** the standard `BENCH.bat` suite, run once per model. Each model's folder has its own report, `results.json` and engine log excerpt. | Model | Chat decode | 4K prompt | 16K prompt | Agent decode | |---|---:|---:|---:|---:| | Q2_0 | 50.2 | 838 | 1,014 | 62.7 | | Swift IQ2_XS | 47.4 | 782 | 968 | 55.9 | | Swift IQ3_XXS | 39.2 | 683 | 946 | 43.3 | | IQ3_XXS | 39.0 | 681 | 940 | 46.4 | | Coder IQ1_M | 38.2 | 923 | 1,126 | 46.4 | | IQ3_S | 33.9 | 599 | 892 | 39.3 | All figures are tok/s. **Also measured:** two flags that setup had added when the machine had less RAM. - Removing `--resident-experts` (Coder IQ1_M): decode +30%, 16K prompts +17%. - Removing `--mmap-experts` (Swift IQ2_XS): decode +34%, 16K prompts +42%. Both "before" runs are in `flag-removal/`. Setup doesn't remove these flags when RAM grows, so older installs that have since gained RAM may be slower than they need to be. **Limitations:** - One benchmark session per model. - Prompts are at most 16K tokens. - Speed only; no recall or quality check. - The reports show a local merge commit: upstream v0.1.45 plus Python-only branches, including the BENCH fix #2919, which BENCH needs in order to start a server.
No site
Links install, modelos, releases.