Issues / #2523

#2523 RX 7900 XTX (gfx1100) on Windows, v0.1.44.1: --adapt-async +6.2%, STRATA_ROUTE_TAIL_SKIP=7 +8.5% (cumulative 64.8 → 79.2 tok/s)

open · @bigbrid2023 · 1 commentaires · Sur GitHub

BenchmarksSetup & installAMD / HIPModels & quantsWindows

Description

Sharing measurements from a Windows + RX 7900 XTX setup, since I understand Windows AMD coverage is limited. All numbers are from the current release, v0.1.44.1, single GPU, IQ3_S pack.

**System**
- GPU: RX 7900 XTX (gfx1100, 24 GB)
- OS: Windows 11 Pro
- CPU / RAM: i9-13900KF, 128 GB DDR4-3200
- Engine: Strata v0.1.44.1 (release build)

**Method**
Fixed fixture for every configuration: 72-token prompt, 256 output tokens, 1 warmup + 3 measured runs, fresh process per run, `STRATA_DECODE_TIMING=1`. Each change was compared against a control re-run in the same batch (batch-to-batch drift on this machine is up to ~±4%). A gain is only reported as confirmed when it exceeds 3% and the run ranges do not overlap.

**Results (median tok/s)**

| Configuration | tok/s | Note |
|---|---|
| Stock defaults (`--spec 4 --spec-min-p 0.5`) | 64.8 | |
| `--spec-min-p 0.7` | 67.1–67.6 | ~+4% |
| + `--adapt-async 1` | 72.4 | +6.2% vs same-batch control (68.2); adapt wait 2.84 → 0.38 ms/window |
| + `STRATA_ROUTE_TAIL_SKIP=7` | 79.2 | +8.5% vs same-batch control (73.0); host CPU experts 4.79 → 2.38 ms/window; VRAM residency 90.7% → 95.1%; MTP acceptance 75.1% → 74.6% |

Two notes that may be useful upstream:

1. **ROUTE_TAIL_SKIP default on gfx1100.** As far as I can tell from the source, the tail-skip default is enabled on HIP only for gfx1200/1201 (`decode_defaults_arch_ok`), and gfx1100 starts with it off. On this card `=7` was worth +8.5% with a negligible acceptance change (−0.5 pt). If the gfx1100 exclusion is not based on a known correctness issue, enabling the same default for gfx1100 would give every 7900 XTX user this gain out of the box. (Understood that outputs differ slightly with it on, as documented.)
2. **Negative results, for the record:** `--kv k8v4` was unstable here (median +7.5%, but one run dropped to 68.8, overlapping the control range, and it added nothing on top of ROUTE_TAIL_SKIP); `--spec 3` was −3.8% vs spec 4; `STRATA_PIN_BY_NEED=1` showed no decode gain. Tuning `--adapt-every / -decay / -swaps` around the defaults also showed no further gain, so the defaults look well chosen.

Happy to provide full timing breakdowns or test other settings on this hardware if useful.

Sur le site

Liens install, modèles, releases.