#5454 [Benchmark] 2x RX 6900 XT (gfx1030), standard protocol on a v0.2.11-based build: 4K / 16K 1,234 / 2,124 (two cards), 1,058 / 1,434 (one card)
closed · @Vop3p · 0 Kommentare · Auf GitHub
BenchmarksMulti-GPUAMD / HIPModels & quantsSecurityDocumentationWindowsLinux
Beschreibung
## Problem and result Two runs of the standard draft protocol (`strata-qwen38-thinking-v1-draft.1`) on the same 2x RX 6900 XT box as #4602 (0.2.7) and #3790 (0.2.2), now with this box's current production engine: v0.2.11 plus a local patch set (details below). Results only, no code or behaviour change requested. Both runs are `complete`, 21/21 required requests, 12 of 12 sanity checks, 0 failed, concurrency included, 0 stalls in the engine log. The data is offered for the community benchmark page; the maintainer noted that newer numbers are welcome after #4602 was closed. | configuration | prompt 4K / 16K tok/s | decode short / 4K / 16K | four requests at once (capacity 1) | |---|---|---|---| | two cards, layer split 27 + on-demand data-parallel prompt read, parallel 1 | **1,234 / 2,124** | 84.7 / 80.7 / 85.5 | 84.5 / 82.5 / 82.8 | | one card | **1,058 / 1,434** | 52.9 / 49.8 / 48.6 | 44.7 / 46.4 / 46.5 | Against the patched 0.2.7 arms of #4602 on the same machine (two cards 1,021 / 1,585, one card 1,023 / 1,232; different day, one pass each): two cards +21% / +34%, one card +3% / +16%. Decode is within the protocol's sampling noise. What changed since #4602 (each step measured on this box with same-session interleaved runs; numbers and method in [dual-6900xt-flash-next-tuning](https://github.com/Vop3p/dual-6900xt-flash-next-tuning/blob/main/docs/linux-dual-6900xt.md)): - v0.2.8 → v0.2.11 merged into the local patch set (each version flat against the one before here). - The data-parallel prompt read from #5002 plus its follow-ups and a few local changes (DP only above 12,288 tokens, so the 4K case is unaffected): the 16K case reads on both cards. - A ggml change that reads the MMQ activations wave-uniformly through the scalar unit on RDNA2 (prompt 4K–128K +2.8~4.9% in this box's synthetic load). - A batch of opt-in kernel changes: the hyper-connection up projection and `attn_qkv` kept in FP16 (bitwise), int8 copies on Q8_0's grid for the verifier's persistent hc read (#2226 + `STRATA_HC_PERSIST_Q8=2`; not bitwise, top-10 overlap with a double-precision reference −0.7 points, KL within chat-to-chat noise), a one-warp dense GEMV for 2–4 tokens and per-format expert layouts for gfx1030 decode (bitwise), the fused SwiGLU+quantize kernel on HIP (byte-identical), upstream's `STRATA_ATTN_MERGE_V2 / FUSE_HITS / IQ_STAGE_TABLES / PLE_COMPACT / PLE_COPY_STREAM`, and `--mtp-q4 head`. ## Related discussion #4602 (the previous report from this box, 0.2.7), #3790 (the 0.2.2 pilot), #5002 (the DP prompt read used by the two-card run), #2226 (the persistent hc read). Single RX 6900 XT reports this complements: #4723, #5320. ## Approach Each server was started from its `config.json` and benchmarked over loopback with the runner from the v0.2.7 checkout (the same runner as #4602): `strata bench run --url http://127.0.0.1:8089 --out <dir> --track custom --track-notes <arm description> --build-options <options> --model-revision ed59f920... --source-revision 4783921b... --optional-concurrency` - Two cards: run `20261107T031700Z-ade4138a`, `"gpu": [1, 0]`, `layer_split` 27, `parallel` 1, no vision encoder; `--prefill 16384 --prefill-pipe 1600 --spec 3 --spec-min-p 0.6 --pipeline-windows 2 --kv int8 --kv-resident 32768 --max-context 262144 --vram-reserve-mib 700 --vram-reserve-later-mib 844 --conversation-cache-mib 0 --mtp-q4 head`; DP env `STRATA_DP_PREFILL=1 STRATA_KV_HOST_DMA=1 STRATA_DP_MIN=12288 STRATA_DP_PEER_RES=0`. - One card: run `20261107T032238Z-96fbb8da`, `"gpu": [1]`, `parallel` 1, no vision encoder; `--prefill auto --prefill-pipe 1600 --spec 3 --spec-min-p 0.6 --pipeline-windows 2 --kv int8 --kv-resident 32768 --max-context 131072 --vram-reserve-mib 700 --conversation-cache-mib 0 --mtp-q4 head`. - Both: run-config sampling temperature 0 / presence penalty 0 (as in #4602); the same env as the production server apart from the vision encoder, including `STRATA_HC_PERSIST=1 STRATA_HC_PERSIST_Q8=2`. The full switch list is in each folder's `config.json` and in the linked document. Source revision `4783921b` is the author's `prod-0211` tree (v0.2.11 merged into the local patch set); the kernel batch is an uncommitted diff on top of it, published as `patches/strata/0002-waitlist-0211-2x6900xt.patch` in the linked repository. Build options: `STRATA_ENABLE_HIP, CMAKE_HIP_ARCHITECTURES=gfx1030, STRATA_NATIVE_EXPERTS, STRATA_PREFILL_MMQ, STRATA_Q6K_EXPERTS, STRATA_MMQ_KQUANTS=OFF, GGML_HIP_GRAPHS, GGML_HIP_NO_VMM`, ggml from llama.cpp `fb5d45967` with the RDNA2 MMQ patches. ## Validation Environment: 2x RX 6900 XT (gfx1030, 16 GB, PCIe 4.0 x8 each, 245 W, −81 mV), Ryzen 5 5600X, 128 GB DDR4-3200, Ubuntu 26.04 kernel 7.0.0-42, ROCm 10.0; model `ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF` IQ3_S revision `ed59f92082b1e93c0e96d60a8b11aab089b52f09`; runner `5e8a4cca7d690ac60ca37eb09e7d0e010099ad6d`; engine 0.1.57; 2026-11-07 03:17 and 03:22 UTC, one engine start per run. **Passed:** - Both runs: complete, 21/21, 12/12 (instruction 4/4, retrieval 4/4, tool calls 4/4), concurrency 3 rounds of 4 requests, 0 failed. - Engine log checks per run: the rocBLAS 5.6.0 table loads (112 rows), `STRATA_HIP_PROMPT_F16=1` and `STRATA_PROMPT_ATTN_RDNA2=1` are confirmed, the two-card 16K case prints `16382 tokens in 5632-token chunks over 2 stages (--prefill-pipe 1600: b/a)`, the hc self-check prints `STRATA_HC_PERSIST_Q8=2 ... max relative difference 0.00497 against the bf16 default read`; no stall reports. - The same engine passed this box's own checks before going to production: deterministic A/B, a whole-run decode A/B with 24 replayed real requests, a teacher-forced KL check against a double-precision reference, 128K / 200K reads and a concurrent worst-case run with the vision encoder. **Failed:** None in the submitted runs. One related note: on one card the int8 hc copies take 641 MiB, and in a separate prompt-speed run (not submitted) one of two cold starts with the same one-card settings refused the 213 MiB draft head (`mtp: the draft head does not fit`, 264 MiB free). The submitted one-card run loaded the draft head normally. One-card users should leave `STRATA_HC_PERSIST_Q8` off or free some VRAM first. **Not run:** - NVIDIA, Metal, SYCL, Windows: not on this box for this build. - Release package / default track on 0.2.11: not repeated; #4602 has them for 0.2.7. - Repeats: one pass per arm. ## Limits One machine, one model and quantization (IQ3_S), one pass per arm, sampling noise in decode. The engine includes local patches that are not in upstream main (#3381, #3382, #3073, #3095, the DP follow-ups, and the kernel batch above); the two-card run is parallel 1 without the vision encoder, unlike this box's production server (parallel 2 + `--batch-mtp` + vision). Developed with an AI coding assistant; every number above was measured on the machine described above. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Mehr auf der Site
Links zu Install, Modellen, Releases.