#2938 Community benchmark: RX 7900 XTX 24 GB (gfx1100), Ryzen 9 9950X, 96 GB DDR5-6400 — 0.1.45 HIP source build, IQ3_S, 262K

Descrição

# Community benchmark on AMD Radeon RX 7900 XTX (gfx1100)

Measured on 2026-10-18 on EndeavourOS (Arch), Linux kernel 7.2.10, ROCm 7.2.4, with a
HIP source build of Strata 0.1.45 and a custom abliterated IQ3_S pack. Single card, one
request at a time.

## Hardware and software

- **GPU:** AMD Radeon RX 7900 XTX (Navi 31, `gfx1100`), 23.98 GiB VRAM;
  **PCIe 4.0 x16** (`LnkSta: Speed 16GT/s, Width x16`). A Radeon 610M iGPU is also present
  and unused.
- **CPU:** AMD Ryzen 9 9950X, 16 cores / 32 threads.
- **RAM:** 96 GB (2×48 GB DDR5-6400, 1DPC, dual channel); `MemTotal` 91.7 GiB.
- **Storage:** SIX X7400 953.9 GB NVMe.
- **OS:** EndeavourOS (Arch), kernel `7.2.10-arch1-1`.
- **Runtime:** ROCm 7.2.4 (`hipcc` 7.2.53211), GCC 16.2.1.
- **Strata:** v0.1.45, commit `477ac13c`; HIP source build (`STRATA_ENABLE_HIP=ON`,
  `STRATA_ENABLE_CUDA=OFF`, `CMAKE_HIP_ARCHITECTURES=gfx1100`, `Release`).
- **Background:** no other GPU workload; desktop idle. No power limit applied.

## Model and configuration

- **Model:** Qwen3.8-Flash-Next, GSQ-RCO-abliterated, IQ3_S native pack.
  - `...-IQ3_S-00001-of-00002.gguf`: 55,063,448,064 bytes
  - `...-IQ3_S-00002-of-00002.gguf`: 28,800,138,432 bytes
- **Vision:** `mmproj-Qwen3.8-Flash-Next-BF16.gguf`, 907,542,944 bytes, `--vision` with the
  encoder on the CPU.
- **Expert profile:** `data/expert-profile.bin` (24,576 ranked pairs).
- **Context:** 262,144. **KV:** `--kv k8v4`, `--kv-resident 32768`.
- **Expert cache:** `--expert-cache 7700` → 9,048 slots, 17.16 GiB VRAM.
- **VRAM reserve:** `--vram-reserve-mib 700`.
- **Prefill:** `--prefill auto:32768`.
- **Conversation cache:** `--conversation-cache-mib 16384`, `--conversation-cache-slots 8`,
  `--conversation-cache-min-tokens 32768`.
- **Speculative:** `--mtp ... --spec 4 --spec-min-p 0.5`.
- **Other:** `--pcie-frac 0`, `--adapt-every 0`, `--pool-workers 15`,
  `--prompt-cache 16 --prompt-cache-every 16384`.
- No calibration run, no experimental speed projection.

```text
strata --serve --pack <pack> --native <IQ3_S-00001-of-00002.gguf>
  --ple-gguf <IQ3_S-00002-of-00002.gguf> --expert-profile data/expert-profile.bin
  --expert-cache 7700 --prefill auto:32768 --spec 4 --spec-min-p 0.5 --mtp <rt>
  --max-context 262144 --kv k8v4 --kv-resident 32768 --vram-reserve-mib 700
  --adapt-every 0 --pcie-frac 0 --pool-workers 15 --vision
  --prompt-cache 16 --prompt-cache-every 16384
  --conversation-cache-slots 8 --conversation-cache-mib 16384
  --conversation-cache-min-tokens 32768
```

## Method

- Prompts: one repeated English paragraph with a unique 2-character tag prefix per run, so
  every run is a **cold** prompt (`0 reused`). Output cap 256 tokens (98K, 260K) or 512
  (short); `temperature: 0`.
- Timing: the engine's own log lines, e.g.
  `prompt 97961 tokens = 0 reused + 97961 read in 54136 ms (1809.5 tok/s), 256 generated in 3253 ms (78.7 tok/s)`.
- Repetitions: 3 runs at 1,457 and 97,961 prompt tokens; the 259,508-token point is a
  **single run** (about 2.5 minutes of prefill each).
- The model was already loaded; loading is not included. Nothing else was running.
- Requests are non-streaming, so **TTFT was not measured by a client**; the prefill
  duration in the table is the closest available proxy (the first token follows prefill
  immediately).
- VRAM was sampled from `/sys/class/drm/card1/device/mem_info_vram_used` during the
  260K run.

## Results

| Configuration | Actual prompt tokens | Reused | Generated | Runs | Prompt tok/s (median, range) | Decode tok/s (median, range) |
| --- | ---: | ---: | ---: | ---: | --- | --- |
| short | 1,457 | 0 | 512 | 3 | **521.0** (494.1–521.5) | **82.5** (82.0–83.5) |
| ~98K | 97,961 | 0 | 256 | 3 | **1809.5** (1795.7–1810.5) | **77.4** (75.8–78.7) |
| ~260K | 259,508 | 0 | 256 | 1 | **1479.6** | **79.9** |

Raw engine lines:

```text
prompt 1457 tokens   = 0 reused + 1457 read in 2794 ms (521.5 tok/s), 512 generated in 6132 ms (83.5 tok/s), drafts accepted 262 of 377
prompt 1457 tokens   = 0 reused + 1457 read in 2796 ms (521.0 tok/s), 512 generated in 6203 ms (82.5 tok/s), drafts accepted 252 of 343
prompt 1457 tokens   = 0 reused + 1457 read in 2949 ms (494.1 tok/s), 512 generated in 6247 ms (82.0 tok/s), drafts accepted 246 of 330

prompt 97961 tokens  = 0 reused + 97961 read in 54108 ms (1810.5 tok/s), 256 generated in 3307 ms (77.4 tok/s), drafts accepted 128 of 166
prompt 97961 tokens  = 0 reused + 97961 read in 54136 ms (1809.5 tok/s), 256 generated in 3253 ms (78.7 tok/s), drafts accepted 129 of 170
prompt 97961 tokens  = 0 reused + 97961 read in 54554 ms (1795.7 tok/s), 256 generated in 3376 ms (75.8 tok/s), drafts accepted 125 of 178

prompt 259508 tokens = 0 reused + 259508 read in 164691 ms (1575.7 tok/s), 256 generated in 3128 ms (81.8 tok/s), drafts accepted 149 of 214
```

**Memory.** VRAM peak 24,545 of 25,753 MiB during the 260K request (as low as 14–18 MiB
free, no failure). RAM: expert arena 46.84 GiB; about 72 GiB used of 91.7 GiB while a
conversation was parked. No OOM, no paging, no stalled window; the engine logged its
"below 512 MiB" warning twice, which is a cross-platform guard (see below).

## Correctness and limitations

- **Not a quality test.** No needle/recall run, no coding task, no tool-call check in this
  report. Speed only.
- **Single machine, single quantization.** IQ3_S only; no other quantization or context
  length, no multi-GPU, no `--batch`.
- **The 260K point is one run.** Prefill at that length takes about 2.5 minutes.
- **`--pcie-frac` costs decode here.** With `--pcie-frac 0.5` (about 11% of experts read
  over PCIe and computed on the GPU) decode measured **48.6 tok/s against 78.3** at
  `--pcie-frac 0`, i.e. -38%, on the same prompt. PCIe 4.0 x16 is roughly 25 GB/s against
  the card's 960 GB/s of VRAM bandwidth, so sending the missed experts to the GPU instead
  of computing them on the CPU is the slower path on this box.
- **`--vram-reserve-mib 700` is tight.** Free VRAM was 368 MiB at request start for 98K and
  236 MiB for 260K. On this Linux/AMD box that produced no slowdown; issue #2065 (NVIDIA
  WDDM paging GPU memory to system RAM, decode collapsing to a few tok/s) did **not**
  reproduce even with 14 MiB free. A larger reserve is safer on other platforms.
- **`--batch 2` gave no throughput gain here.** Two concurrent 1.5K-token requests decoded
  in 12.8 s serial vs 12.8 s batched (512 windows at 25.05 ms each, `avg 1.99 rows`):
  strictly linear, compute-bound. Two 98K prompts were prefill-serialized.
- Prompt throughput at 1,457 tokens (521 tok/s) is lower than at 98K (1,810 tok/s) because
  the fixed per-request overhead dominates at that size.

## Correction

An earlier revision of this report said the card runs at PCIe **x8**. That was wrong: it
came from reading `current_link_width` over every PCI device with an AMD vendor ID, which
also matched an upstream PCIe bridge (`01:00.0`, class `0x060400`, x8) and the iGPU. The
GPU's own link is `Speed 16GT/s, Width x16`. The `--pcie-frac` note above was rewritten
accordingly.