Pull requests / #2705
#2705 bench: community report, RX 7900 XT 20 GB (gfx1100) on Linux, UD-Q5_K_XL, v0.1.44.1
open · @loeeeee · 0 commentaires · Sur GitHub
BenchmarksSetup & installServer & APIAMD / HIPNVIDIA / CUDAModels & quantsDocumentationLinux
Description
## Title bench: community report, RX 7900 XT 20 GB (gfx1100) on Linux, UD-Q5_K_XL, v0.1.44.1 Issue: no issue to resolve. Related: #2065 and #2477 (VRAM headroom), #2505 / #2632 (the UD-Q5_K_XL shard check this install hit). ## Summary One RX 7900 XT 20 GB (gfx1100) in a Linux container on an EPYC 7K62 (48 CPUs, 128 GiB), v0.1.44.1 source build on a system ROCm 7.2.3 (hipBLASLt 1.2.2, setup picked the `100202` table), Unsloth UD-Q5_K_XL at 262,144 tokens with every expert in RAM. Hardware the maintainers do not list: a 7900 XT (not XTX) on Linux. Same build, same machine, same day, `tools/strata_bench.py --runs 3 --long`, setup's configuration vs the same plus `--vram-reserve-mib 1024 --prefill auto:32768` (medians): | Test | setup default | tuned | | --- | ---: | ---: | | short chat decode tok/s | 17.2 | 22.7 | | prompt tok/s, 4K / 16K / 32K fresh | 335 / 565 / 1,019 | 577 / 888 / 1,116 | | agent follow-up first token s | 7.17 | 5.36 | | agent follow-up decode tok/s | 15.1 | 26.1 | `needle_bench.py` on the tuned configuration: 6 of 6 at 32K and 125K tokens (depths 10/50/90 %). With setup's configuration only 562 MiB of VRAM was free after start and `serve` printed the `below 512 MiB` warning; the reserve trades 85 GPU expert slots for 888 MiB free. In the screening runs `STRATA_PREFILL_MMQ=1` was slower or no faster in every pairing (unlike the IQ3_XXS / ROCm 7.1 numbers in `docs/AMD_HIP_PERFORMANCE.md`), as were `--pool-workers 47` and `--adapt-every 0`. One screening arm lost its run to `verify: timed out at layer 17` (#267 wording) after a stall; no failure in an 8-round soak of the kept configuration or in either suite run. ## What changed - `bench/results/2026-10-15-community-rx-7900-xt-ud-q5-k-xl/`: `README.md` (hardware, ROCm, configuration, method, results, failures, what was not tested); `setup-default/` and `tuned/` (the bench tool's own `README.md` + `results.json` for each run); `needles.json`; `micro/` (the deterministic screening script and every arm's settings and samples, 15 KB). - `bench/results/COMMUNITY.md`: one row. - `docs/COMMUNITY_BENCHMARKS.md`: one entry. No code changes. ## Extra Notes - Install: one local change to `setup.py` only, not to the engine: `check_shards` skips shards with no tensors, because setup refused the current UD-Q5_K_XL `00001-of-00006` (10,946,618 bytes, identical to setup's own sha256 pin) as 6 bytes short. Same fix as #2505 / #2632; not part of this PR. - The tool READMEs say "Settings: the install's own configuration, unchanged" because the tool was pointed at a running server; the top-level README states which configuration each folder ran. - Not tested: thinking on, concurrency / `--batch`, vision, prompts above ~125K tokens, other quants, `--calibrate`, power and clocks. The host's other guests were not stopped and the container's CPUs are not pinned to whole cores. - Measured and written up with an AI coding agent (Claude Code); numbers are from the files in the folder.
Sur le site
Liens install, modèles, releases.