Issues / #816

#816 0.1.39: decode on an RX 6800 (gfx1030, Windows) is 12-22% slower than 0.1.38; STRATA_SH_STREAM=0 removes it

0.1.39: decode on an RX 6800 (gfx1030, Windows) is 12-22% slower than 0.1.38; ST

closed · @did-technomancer · 12 commentaires · Sur GitHub

BenchmarksSetup & installAMD / HIPModels & quantsWindows

Description

```text
GPU      AMD Radeon RX 6800 16 GB (gfx1030)
CPU      AMD Ryzen 7 5700X3D
RAM      64 GB DDR4-3200
OS       Windows 11 26H2 (build 26300)
Driver   AMD 32.0.21043.10005
ROCm     10.2.0a20260930
Model    IQ2_XS, 128K context, setup's configuration unchanged
```

- 0.1.38: release `strata-windows-x64-hip.zip`; built here the same way as 0.1.39, it gave the same numbers
- 0.1.39: tag `v0.1.39` built here with `tools\hip\build_windows.bat`, `STRATA_HIP_ARCHS=gfx1030`; the release zip was
  not run
- Decode: tok/s from the engine's timings, each value a separate run

| Engine | English | Ukrainian | Code | Prompt 30.3K |
| --- | --- | --- | --- | ---: |
| 0.1.38 | 50.5 / 55.2 | 40.8 / 42.3 | 57.2 / 55.8 | 309 / 310 |
| 0.1.39 | 42.6 / 42.5 | 32.9 / 32.5 | 49.2 / 50.2 | 310 / 311 |
| 0.1.39 + `STRATA_SH_STREAM=0` | 56.4 / 57.0 / 55.5 / 54.2 | 43.5 / 44.3 / 43.2 / 43.0 | 62.7 / 62.9 / 60.5 / 61.5 | 311 |

0.1.39's new decode switches, turned off one at a time (1 run each):

| Changed | English | Ukrainian | Code |
| --- | ---: | ---: | ---: |
| nothing | 42.5 | 32.5 | 50.2 |
| `STRATA_SH_STREAM=0` | 56.4 | 43.5 | 62.7 |
| `STRATA_IQ_STAGE_GRID=0` | 43.5 | 34.6 | 51.8 |
| `STRATA_PLE_BATCH=0` | 41.6 | 33.0 | 49.0 |
| `STRATA_OLD_IQ_MMVQ=1` | 43.3 | 32.5 | 47.9 |
| `STRATA_DEC_BATCH=0` | 40.0 | 31.8 | 45.4 |

On IQ3_XXS (`--draft-vocab cyrillic`) 0.1.39 + `STRATA_SH_STREAM=0` gives 49.2 / 55.2 / 52.0 against 42.2 / 45.7 / 42.8
on 0.1.38. Why the shared expert's second stream costs this much here was not profiled.

Full report, scripts and data: https://github.com/Niko1221/Strata/pull/815

Sur le site

Liens install, modèles, releases.