Pull requests / #2705

#2705 bench: community report, RX 7900 XT 20 GB (gfx1100) on Linux, UD-Q5_K_XL, v0.1.44.1

open · @loeeeee · 0 comments · View on GitHub

BenchmarksSetup & installServer & APIAMD / HIPNVIDIA / CUDAModels & quantsDocumentationLinux

Description

## Title
bench: community report, RX 7900 XT 20 GB (gfx1100) on Linux, UD-Q5_K_XL, v0.1.44.1

Issue: no issue to resolve. Related: #2065 and #2477 (VRAM headroom), #2505 / #2632 (the UD-Q5_K_XL shard check this install hit).

## Summary
One RX 7900 XT 20 GB (gfx1100) in a Linux container on an EPYC 7K62 (48 CPUs, 128 GiB), v0.1.44.1 source build on a system ROCm 7.2.3 (hipBLASLt 1.2.2, setup picked the `100202` table), Unsloth UD-Q5_K_XL at 262,144 tokens with every expert in RAM. Hardware the maintainers do not list: a 7900 XT (not XTX) on Linux.

Same build, same machine, same day, `tools/strata_bench.py --runs 3 --long`, setup's configuration vs the same plus `--vram-reserve-mib 1024 --prefill auto:32768` (medians):

| Test | setup default | tuned |
| --- | ---: | ---: |
| short chat decode tok/s | 17.2 | 22.7 |
| prompt tok/s, 4K / 16K / 32K fresh | 335 / 565 / 1,019 | 577 / 888 / 1,116 |
| agent follow-up first token s | 7.17 | 5.36 |
| agent follow-up decode tok/s | 15.1 | 26.1 |

`needle_bench.py` on the tuned configuration: 6 of 6 at 32K and 125K tokens (depths 10/50/90 %).

With setup's configuration only 562 MiB of VRAM was free after start and `serve` printed the `below 512 MiB` warning; the reserve trades 85 GPU expert slots for 888 MiB free. In the screening runs `STRATA_PREFILL_MMQ=1` was slower or no faster in every pairing (unlike the IQ3_XXS / ROCm 7.1 numbers in `docs/AMD_HIP_PERFORMANCE.md`), as were `--pool-workers 47` and `--adapt-every 0`. One screening arm lost its run to `verify: timed out at layer 17` (#267 wording) after a stall; no failure in an 8-round soak of the kept configuration or in either suite run.

## What changed
- `bench/results/2026-10-15-community-rx-7900-xt-ud-q5-k-xl/`: `README.md` (hardware, ROCm, configuration, method, results, failures, what was not tested); `setup-default/` and `tuned/` (the bench tool's own `README.md` + `results.json` for each run); `needles.json`; `micro/` (the deterministic screening script and every arm's settings and samples, 15 KB).
- `bench/results/COMMUNITY.md`: one row.
- `docs/COMMUNITY_BENCHMARKS.md`: one entry.

No code changes.

## Extra Notes
- Install: one local change to `setup.py` only, not to the engine: `check_shards` skips shards with no tensors, because setup refused the current UD-Q5_K_XL `00001-of-00006` (10,946,618 bytes, identical to setup's own sha256 pin) as 6 bytes short. Same fix as #2505 / #2632; not part of this PR.
- The tool READMEs say "Settings: the install's own configuration, unchanged" because the tool was pointed at a running server; the top-level README states which configuration each folder ran.
- Not tested: thinking on, concurrency / `--batch`, vision, prompts above ~125K tokens, other quants, `--calibrate`, power and clocks. The host's other guests were not stopped and the container's CPUs are not pinned to whole cores.
- Measured and written up with an AI coding agent (Claude Code); numbers are from the files in the folder.

Related on this site

Quick links to related pages.