Pull requests / #2827
#2827 bench: RX 7900 XTX (gfx1100) on AOOSTAR AG03 Oculink eGPU — swift-1.5 IQ2_XS, 131K context
open · @kurotope · 0 Kommentare · Auf GitHub
BenchmarksSetup & installAMD / HIPModels & quantsDocumentation
Beschreibung
Adds a community benchmark: AMD Radeon RX 7900 XTX 24 GB (gfx1100) in an AOOSTAR AG03 eGPU dock (Sapphire Pulse) over Oculink, driven by an ASRock NUC Box K12 mini-PC (Ryzen 7 H 255, 64 GB RAM, Fedora). ## Setup - **Strata** 0.1.44.1 (commit 296b59d2), engine 0.1.44.1, hip backend - **Model** swift-1.5, IQ2_XS, 131,072-token context, 1 parallel slot - **Config** `--spec 4 --spec-min-p 0.5 --mtp rt --kv int8 --kv-resident 32768`, thinking off, temperature 0 - **GPU connection:** the 7900 XTX is not socketed in the mini-PC — it's an eGPU over Oculink. The engine matches the link at `pcie_frac 0.19` (~1/4 of a desktop x16 PCIe 4.0 bandwidth), so prompt/decode transfers are Oculink-bound. Noted in the report so reviewers don't discount the prompt rates. ## Results (suite 1, 3 runs each, medians with ranges) | Test | Prompt tok/s | Decode tok/s | TTFT | | --- | ---: | ---: | ---: | | Short chat (~46 → 100 new tokens) | — | 86.1 (67.1–105.3) | 0.35 s | | 4K fresh | 1098.4 | — | 3.68 s | | 16K fresh | 1389.7 | — | 11.58 s | | 32K fresh | 1411.9 | — | 22.74 s | | Agent follow-up turns | ~800 cached | 85.4 (68.1–122.5) | 3.65 s | - Agent-session conversation-cache hit: **68.6%**, only ~52 earlier tokens re-read per session; ~13,729 new file tokens read once. ## Notes / limitations - Speed only — no correctness check was run. - Background load and GPU power limits not recorded. - Suite total: 206 s. No concurrency test (`--clients`) was run. - Per-request data: `results.json`. The server was already running, so there is no `engine-log-excerpt.txt` in this folder. Adds `bench/results/2026-10-17-community-rx-7900-xtx-oculink-swift-1-5-iq2-xs-2/` (README.md + results.json). No engine or runtime changes.
Mehr auf der Site
Links zu Install, Modellen, Releases.