改动 / #2827

#2827 bench: RX 7900 XTX (gfx1100) on AOOSTAR AG03 Oculink eGPU — swift-1.5 IQ2_XS, 131K context

open · @kurotope · 0 评论 · 去 GitHub 看

BenchmarksSetup & installAMD / HIPModels & quantsDocumentation

说明

Adds a community benchmark: AMD Radeon RX 7900 XTX 24 GB (gfx1100) in an
AOOSTAR AG03 eGPU dock (Sapphire Pulse) over Oculink, driven by an ASRock
NUC Box K12 mini-PC (Ryzen 7 H 255, 64 GB RAM, Fedora).

## Setup

- **Strata** 0.1.44.1 (commit 296b59d2), engine 0.1.44.1, hip backend
- **Model** swift-1.5, IQ2_XS, 131,072-token context, 1 parallel slot
- **Config** `--spec 4 --spec-min-p 0.5 --mtp rt --kv int8 --kv-resident 32768`,
  thinking off, temperature 0
- **GPU connection:** the 7900 XTX is not socketed in the mini-PC — it's an
  eGPU over Oculink. The engine matches the link at `pcie_frac 0.19`
  (~1/4 of a desktop x16 PCIe 4.0 bandwidth), so prompt/decode transfers
  are Oculink-bound. Noted in the report so reviewers don't discount the
  prompt rates.

## Results (suite 1, 3 runs each, medians with ranges)

| Test | Prompt tok/s | Decode tok/s | TTFT |
| --- | ---: | ---: | ---: |
| Short chat (~46 → 100 new tokens) | — | 86.1 (67.1–105.3) | 0.35 s |
| 4K fresh | 1098.4 | — | 3.68 s |
| 16K fresh | 1389.7 | — | 11.58 s |
| 32K fresh | 1411.9 | — | 22.74 s |
| Agent follow-up turns | ~800 cached | 85.4 (68.1–122.5) | 3.65 s |

- Agent-session conversation-cache hit: **68.6%**, only ~52 earlier tokens
  re-read per session; ~13,729 new file tokens read once.

## Notes / limitations

- Speed only — no correctness check was run.
- Background load and GPU power limits not recorded.
- Suite total: 206 s. No concurrency test (`--clients`) was run.
- Per-request data: `results.json`. The server was already running, so there
  is no `engine-log-excerpt.txt` in this folder.

Adds `bench/results/2026-10-17-community-rx-7900-xtx-oculink-swift-1-5-iq2-xs-2/`
(README.md + results.json). No engine or runtime changes.

本站相关内容

相关页面的快捷入口。