Pull requests / #3349

#3349 [Benchmark] RX 7900 XT 20 GB, UD-Q5_K_XL: v0.1.44.1 vs v0.2.0-beta

open · @loeeeee · 0 commentaires · Sur GitHub

BenchmarksSetup & installAMD / HIPDocumentationLinux

Description

## Problem and result
Results-only benchmark report from an RX 7900 XT 20 GB (gfx1100) on Linux with Unsloth UD-Q5_K_XL at 262K, all experts in RAM. This PR was closed in the 0.2.0-beta tracker reset; I rebased it on current `main` and re-ran the measurements on v0.2.0-beta as the reset message asked. It now carries two reports:

- `2026-10-15-community-rx-7900-xt-ud-q5-k-xl` (v0.1.44.1): setup's default configuration vs the same plus `--vram-reserve-mib 1024 --prefill auto:32768`, same day. Unchanged from the original submission.
- `2026-10-20-community-rx-7900-xt-ud-q5-k-xl-0.2.0-beta` (new): the tuned configuration measured the same day on v0.1.44.1 and then on v0.2.0-beta (engine 0.1.45.3), `tools/strata_bench.py --runs 3 --long`, medians:

| | v0.1.44.1 | v0.2.0-beta |
| --- | --- | --- |
| 4K / 16K / 32K prompt, tok/s | 226 / 833 / 902 | 828 / 1,153 / 1,050 |
| short-chat decode, tok/s | 16.2 | 18.6 |
| agent follow-up first token, s | 5.79 | 5.67 |

Prompt throughput at 16K and 32K is higher and steadier on v0.2.0-beta (+38 % / +16 % in the medians); decode and agent turns are not different beyond the run ranges. The v0.1.44.1 4K run was noisy (172-311 tok/s) and well below the 2026-10-15 result on the same build (577), so the 4K difference is not a speedup claim.

## Related discussion
Continues #2705, which the maintainer closed in the 0.2.0-beta reset ("closed here does not mean rejected"). GitHub would not reopen it after the rebase, so this is the same branch as a new PR. Results-only, as `docs/COMMUNITY_BENCHMARKS.md` describes; no engine changes.

## Approach
New report folder with a comparison README, the tool's own README and `results.json` for each build, one `COMMUNITY.md` row and one `docs/COMMUNITY_BENCHMARKS.md` entry. The tool's output was edited in two places only: the GPU name (it printed "AMD Radeon") and the service user name that its sanitizer replaced in the Strata version line.

## Validation
**Passed:**
- `python tools/strata_bench.py --url http://127.0.0.1:8080 --runs 3 --long` against the running server, once on v0.1.44.1 and once on v0.2.0-beta, same machine and configuration, 2026-10-20. No failed requests in either. Per-request data in each folder's `results.json`.
- Update to v0.2.0-beta via `setup.sh --backend hip` (source build, ROCm 7.2.4): server loaded, engine log "session is up (engine 0.1.45.3)".
- Hand-run pilot of `docs/BENCHMARK_PROTOCOL.md` (draft; schedule, settings, fixtures and scoring followed by hand with `protocol-pilot/scripts/pilot.py`), two complete runs on a freshly restarted engine each, v0.2.0-beta only: **12/12 sanity checks in both runs** (4 JSON, 4 retrieval at 4K/16K, 4 tool calls), no failed requests, 0 reused prompt tokens in the performance requests, 7 min required stages. Optional concurrency on the one-slot server (queued, aggregate 18-20.5 tok/s, equal to one stream).
- `tools/needle_bench.py` at 32K and 124K prompts, depths 10/50/90: 6/6 found in both runs.
- Memory/clock telemetry sampled every ~2 s across run 2 (574 rows): VRAM 19.3 of 20.0 GiB steady, host RAM 97-98.5 of 128 GiB, shader clock median 2.56 GHz, PCIe Gen4 x16, power median 210 W.

**Failed:**
- First v0.2.0-beta setup link failed (`undefined reference: sqrtf@GLIBC_2.43` from librocblas) because the CMake cache in `build-hip` still pointed at the previous ROCm (7.2.3) and glibc. Moving `build-hip` aside and rerunning fixed it. Noted in the report; I did not investigate whether setup should detect a changed ROCm root.
- A second setup run failed copying into `engine/strata` ("Text file busy") while the server was running; stopping the server first fixed it.

**Not run:**
- The draft protocol as a conforming run: no runner, approved fixture bundle or JSON schema exists yet, so the pilot is explicitly nonconforming (deviations listed in `protocol-pilot/README.md`: unpinned tokenizer/template, resolved server config and seed handling not verified, model file hashes not recorded, flat result structure).
- The protocol pilot on v0.1.44.1, so the pilot supports no before/after claim; 262K prompts; telemetry for run 1 (sampler PATH bug, fixed for run 2); other backends (this is one AMD box, and no engine change is proposed, so no builds beyond the HIP one above).

## Limits
The before/after comparison is one session per build, not a controlled A/B; the protocol pilot (v0.2.0-beta only, two replicate runs on the same machine) is nonconforming and not a hardware replication. The OS update moved nixpkgs ROCm 7.2.3 -> 7.2.4 between the two runs, so the Strata tag and ROCm are changed together. The machine is shared with coding agents, background load was not controlled, and the cause of the baseline's noise is unknown.

Sur le site

Liens install, modèles, releases.