Pull requests / #2749

#2749 Add community benchmark: 4x AMD Radeon gfx1201, EPYC 7452, UD-Q5_K_XL

open · @TechwriterSSchmidt · 0 Kommentare · Auf GitHub

BenchmarksSetup & installMulti-GPUAMD / HIPModels & quantsDocumentationLinux

Beschreibung

## Summary
Community benchmark run with the official `tools/strata_bench.py --long` suite (3 runs per test, medians) on:

- **4x AMD Radeon (gfx1201, Device 0x7551), 127 GiB VRAM total**
- AMD EPYC 7452 (32C/64T), 220 GiB RAM, Linux (Fedora kernel 7.2.10), HIP
- Strata 0.1.44.1 (commit 296b59d2), UD-Q5_K_XL, 262,144-token context, layer split across 4 GPUs, every expert in RAM

Headline numbers (medians): short-chat decode **45.1 tok/s**; fresh prompts **1,228 / 2,048 / 2,687 tok/s** at 4K / 16K / 32K (TTFT 3.31 / 7.89 / 12.0 s); agent session **68.6 % cache hit**, 44.9 tok/s decode, 74 re-read tokens per session.

Ran against the already-running production server (`--url`), so model loading is not timed and the configuration is the install's own, unchanged. Speed only, no correctness run.

## Notes
- `rocm-smi` does not report a product name on this box, hence "AMD Radeon (gfx1201, Device 0x7551)" in the title; the folder name keeps the tool-generated `4x-unknown` slug.
- README and `results.json` are the tool's sanitized output (no host names, IPs or local paths).

## Test plan
- [x] `strata_bench.py --long` completed cleanly (exit 0, 316 s)
- [x] Output sanitized per docs/COMMUNITY_BENCHMARKS.md

Mehr auf der Site

Links zu Install, Modellen, Releases.