#5446 [Benchmark] 2x RX 6800 XT (x8 + x8) on Windows / UD-Q4_K_XL / Strata 0.2.11 / engine 0.1.57

closed · @Expired-Pasta · 0 コメント · GitHub で見る

BenchmarksSetup & installServer & APIMulti-GPUAMD / HIPNVIDIA / CUDAModels & quantsSecurityDocumentationWindowsLinux

本文

# Benchmark report — 2x RX 6800 XT (x8 + x8): UD-Q4_K_XL on Strata 0.2.11 / engine 0.1.57

Draft protocol `strata-qwen38-thinking-v1-draft.1`, run `20261107T013408Z-717a22f7`.

This adds `bench/results/2026-11-07-standard-rx-6800-xt-plus-rx-6800-xt-717a22f7/` only — no engine or configuration changes.

This is a follow-up to [#4868](https://github.com/Niko1221/Strata/pull/4868), on the same machine with the same
UD-Q4_K_XL model. Since that report I moved the second card from the chipset x4 slot to the CPU x8 slot and updated
to Strata 0.2.11 / engine 0.1.57. It is a combined engine + PCIe topology result, not an isolated engine comparison.

> **Naming note:** the Strata engine log calls the two HIP devices `CUDA0` and `CUDA1`. On this AMD machine these are
> device ordinals, not NVIDIA CUDA; both cards are Radeon RX 6800 XT (gfx1030, HIP). “CUDA1” below means the second
> Radeon card.

## Problem and result

RDNA2 on Windows is still a thin part of the benchmark set, and this machine is an unusual budget dual-GPU setup:
two RX 6800 XT cards, 128 GB of DDR4-3200, and a 103.7 GB 4-bit model that streams experts from RAM.

Before this run, the only standard-protocol Q4 result from this machine was #4868: 0.2.10 / 0.1.56, **x16 + x4**
(measured 27.9 / 6.3 GB/s), K=42, decode 29.6 / 26.4 / 28.5 tok/s, prompt 331.3 / 460.6 tok/s, sanity 12/12.

After moving the second card to x8 and updating the engine, the same profile measures decode **34.1 / 35.8 / 31.8
tok/s** and prompt **226.1 / 354.9 tok/s**, sanity **12/12**. Decode improves 12–36%; prompt and TTFT regress 23–32%
because the old x16 card no longer carries most of the prefill. The full suite is in the result folder.

## Related discussion

- #4868 — previous standard Q4 report from this machine (0.2.10 / 0.1.56, x16 + x4, K=42). The maintainer collected
  those results for the website and invited a new 0.2.11 report; this is that report.
- No engine issue or code change is proposed here.

## Approach

- Existing profile: `D:\Strata\launch\ud-q4_k_xl-32k.json` (32,768-token context, `--kv int8`,
  `--expert-cache auto`, `--prefill auto`, `--vision` CPU encoder, one server slot).
- `strata bench run --profile ...` started the engine, waited for the model, ran the standard suite, and stopped the
  engine it started. The engine log in the folder is the first-hand record of the run.
- The actual engine arguments show **DFlash2** (`--draft dflash2 --spec 6 --draft-conf 0.35`,
  `--draft-fallback none`), not MTP, and `--layer-split auto` chose **K=25**.
- The 0.1.57 engine package used the existing profile's 0.1.56 expert-profile data and vision binary. The engine
  binary and inference path were not modified.

Packaging details, both to keep the artifacts verifiable:

- the folder's `.gitattributes` sets `* -text`, so `SHA256SUMS.txt` matches the committed bytes on any checkout;
- `engine.log` is added with `git add -f` because the repository ignores `*.log`; it is the only first-hand record
  of the PCIe probe, K=25, the expert-cache slot count and the DFlash2 startup;
- `SHA256SUMS.txt` lists every file in the folder (11 files).

One setup note for other Windows users: 0.2.11's setup still refuses **new** multi-AMD profiles on Windows
("several AMD cards sharing one model (--gpus) is Linux-only for now"). The already-configured profile starts
through `strata bench run --profile` / `strata serve --profile`, which is how this run was made. This is a setup
guard, not an engine limit; no local patch was needed for this run.

## Validation

**Passed:**

- `strata bench run --profile "D:\Strata\launch\ud-q4_k_xl-32k.json" --track custom --track-notes "Strata 0.2.11 / engine 0.1.57 / 2x RX 6800 XT 16GB / PCIe 4.0 x8+x8 / K=25 auto / Ryzen 9 3950X / 128GB DDR4-3200 / UD-Q4_K_XL 32K / kv int8 / DFlash2 / vision on" --output human`
  on Strata 0.2.11 / engine 0.1.57 hip-rdna2 (`c3b3f50`), Windows 11 Home 10.0.26300, Ryzen 9 3950X,
  2 x RX 6800 XT (gfx1030), driver 32.0.21045.16010.
  Result: 21 measured requests + 1 warm-up, **sanity 12/12** (instruction 4/4, retrieval 4/4, tool calls 4/4);
  decode 34.1 tok/s (short), 35.8 (4K), 31.8 (16K); prompt 226.1 tok/s (4K), 354.9 (16K);
  8 min 59 s measured. Log: `engine.log` and `results.json` in the folder.
- **PCIe probe:** both cards measured 14.0 GB/s host→device; the engine selected K=25 and reported 95.7% of the
  routed mass in the expert caches. The second device (`CUDA1` in the engine log; HIP device 1) had 2,018 MiB of
  VRAM free after load (no 0 MiB / paging warning).
- **SHA256:** `SHA256SUMS.txt` covers all 11 files in the folder; the hashes were checked locally against the
  files before staging.

**Failed:**

none observed.

**Not run:**

- Engine build and unit tests: this PR changes no code, only adds result files.
- CUDA / SYCL / Metal and Linux HIP builds: not applicable to an artifact-only change and not available on this
  Windows machine.
- The optional concurrency extension (12 requests at concurrency 4): the server was configured with one slot,
  so it would only have measured queueing.
- Background desktop work was not measured; the quantization revision was not read.

## Limits

- Draft protocol; single pass with no repetitions. Run-to-run variance on this machine is visible, so the medians
  are one sample, not a precise speedup claim.
- 6 of the 9 performance requests reached the 512-token cap while still reasoning (thinking, effort xhigh); the
  16K case has no first-answer time.
- Warm-cache measurement: the expert cache and PLE table were warm; this is not a cold-model run.
- **Engine and PCIe topology changed together.** The decode/prompt differences cannot be attributed to 0.1.57
  alone. The x8/x8 change is the dominant explanation for the balanced K=25 split and the prompt shift.
- The runner's hardware block prints driver 32.0.21045.11001; Windows reports 32.0.21045.16010 for both cards.

## Hardware

| Item | Value |
|---|---|
| CPU | AMD Ryzen 9 3950X (16C/32T) |
| Motherboard | ASRock X570 Taichi, BIOS P5.80 |
| RAM | 128 GB (4 x 32 GB), DDR4-3200 |
| GPU | 2 x Radeon RX 6800 XT 16 GB (gfx1030), driver 32.0.21045.16010 |
| PCIe | **both cards x8 (14.1 / 14.0 GB/s measured host→device); the second card is no longer in the chipset x4 slot** |
| Storage | NVMe (model files on a WDC WDS500G2B0C) |
| OS | Windows 11 Home 10.0.26300 |

## Configuration

- 32,768-token context, `--kv int8`, `--expert-cache auto`, `--prefill auto`, `--vision` (CPU encoder),
  `--layer-split auto` (K=25), one server slot.
- Drafter: **DFlash2** (`--draft dflash2 --spec 6 --draft-conf 0.35`, `--draft-fallback none`).
- PLE table (26.8 GiB) resident in RAM (`--ple-io ram`); the profile also sets `STRATA_ROUTE_TAIL_SKIP=0`.
- Device 0 (`CUDA0` in the engine log) runs layers 0–24; device 1 (`CUDA1` in the log) runs layers 25–47 plus the
  head, with a 2,174-slot / 6.35 GiB expert cache. The CUDA-style names come from the engine's HIP backend, not
  from NVIDIA hardware.

## Results (21 measured requests + 1 warm-up; 12/12 sanity)

| Case | Prompt tok/s | Decode tok/s | TTFT | Total |
|---|---:|---:|---:|---:|
| short (~80 tokens) | – | 34.1 (30.6–37.4) | 4.25 s | 13.4 s |
| 4K | 226.1 (223.0–227.2) | 35.8 (34.3–37.9) | 18.16 s | 32.4 s |
| 16K | 354.9 (354.9–359.1) | 31.8 (31.2–38.3) | 46.23 s | 62.3 s |

Sanity checks: 12 of 12 passed (instruction 4/4, retrieval 4/4, tool calls 4/4).

## Comparison with #4868

Same machine, same model, same 32K / int8 / DFlash2 settings. The difference is the engine (0.1.56 → 0.1.57) and
the PCIe topology (x16 + x4 → x8 + x8), which also moved the split from K=42 to K=25.

| Case | #4868: 0.2.10, x16+x4, K=42 | **This run: 0.2.11, x8+x8, K=25** | Change |
|---|---:|---:|---:|
| short decode | 29.6 tok/s | **34.1 tok/s** | +15% |
| short TTFT | 3.33 s | **4.25 s** | slower |
| 4K prompt | 331.3 tok/s | **226.1 tok/s** | **−32%** |
| 4K decode | 26.4 tok/s | **35.8 tok/s** | **+36%** |
| 4K TTFT | 12.42 s | **18.16 s** | slower |
| 16K prompt | 460.6 tok/s | **354.9 tok/s** | **−23%** |
| 16K decode | 28.5 tok/s | **31.8 tok/s** | **+12%** |
| 16K TTFT | 35.64 s | **46.23 s** | slower |
| 4K total | 31.8 s | **32.4 s** | ~same |
| 16K total | 53.6 s | **62.3 s** | slower |
| Sanity | 12/12 | **12/12** | same |

## What may interest other users

- **x8/x8 changes the tradeoff, not just the speed.** The slower card's link doubles (6.3 → 14.0 GB/s), the split
  becomes balanced (K=25: 25 / 23 layers), and decode rises 12–36%. The prompt path loses the single x16 card that
  used to carry most of the prefill, so prompt throughput and TTFT regress.
- **Expert-cache coverage is high:** the caches hold 6,216 of 24,576 profiled pairs, about 95.7% of the routed mass.
  The second Radeon card (`CUDA1` in the engine log) keeps 2,018 MiB free after load; this run shows none of the
  0 MiB / WDDM paging the previous topology could produce.
- **For coding/agent work, TTFT matters.** A fresh 16K prompt now takes 46 s instead of 36 s; conversation-cache
  reuse softens this in real sessions, but the standard suite uses fresh prompts.
- **0.2.11 is stable here on 2 x 16 GB.** The release's small-multi-card steps (trim, KV step-down, drafter drop,
  pinned head) were not needed in the log; no `short of GPU memory` line appears.

## Files

`bench/results/2026-11-07-standard-rx-6800-xt-plus-rx-6800-xt-717a22f7/` contains:

`README.md` (runner report), `results.json`, `outputs.jsonl`, `config.json`, `provenance/`, `engine.log`
(that session only), `engine-argv.txt`, `NOTES.md` (English + Korean author notes), `SHA256SUMS.txt` and
`.gitattributes`.

No credentials, prompts from other work, or personal paths are included; the machine name does not appear in the
engine log.

---

# 벤치마크 보고서 — RX 6800 XT 2장 (x8 + x8), UD-Q4_K_XL, Strata 0.2.11 / 엔진 0.1.57

드래프트 프로토콜 `strata-qwen38-thinking-v1-draft.1`, 실행 `20261107T013408Z-717a22f7`.

`bench/results/2026-11-07-standard-rx-6800-xt-plus-rx-6800-xt-717a22f7/` 폴더만 추가하며, 엔진·설정 변경은 없습니다.

같은 머신·같은 UD-Q4_K_XL 모델로 올리는 [#4868](https://github.com/Niko1221/Strata/pull/4868)의 후속 보고서입니다.
그때 이후 두 번째 카드를 칩셋 x4 슬롯에서 CPU x8 슬롯으로 옮기고 Strata 0.2.11 / 엔진 0.1.57로 업데이트했습니다.
따라서 엔진과 PCIe 토폴로지가 함께 바뀐 결과이며, 엔진 단독 비교가 아닙니다.

> **용어 참고:** Strata 엔진 로그는 HIP 디바이스도 `CUDA0`, `CUDA1`로 표기합니다. 이 머신에서 이는 장치 번호일
> 뿐 NVIDIA CUDA가 아니며, 두 카드 모두 Radeon RX 6800 XT(gfx1030, HIP)입니다. 아래의 “CUDA1”은 **두 번째
> Radeon 카드**를 뜻합니다.

## 문제와 결과

RDNA2의 Windows 자료는 여전히 적고, 이 머신은 RX 6800 XT 2장, DDR4-3200 128 GB, 103.7 GB 4-bit 모델을 쓰는
특이한 구성입니다. 이 머신의 마지막 표준 Q4 결과는 #4868(0.2.10 / 0.1.56, **x16 + x4**, K=42)로,
디코드 29.6 / 26.4 / 28.5 tok/s, 프롬프트 331.3 / 460.6 tok/s, 정합성 12/12였습니다.

두 번째 카드를 x8로 옮기고 엔진을 올린 뒤 같은 프로파일에서 디코드 **34.1 / 35.8 / 31.8 tok/s**,
프롬프트 **226.1 / 354.9 tok/s**, 정합성 **12/12**를 기록했습니다. 디코드는 12~36% 올랐고, 프롬프트와 TTFT는
23~32% 느려졌습니다. x16 카드가 더 이상 prefill 대부분을 담당하지 않기 때문입니다.

## 관련 논의

- #4868 — 이 머신의 이전 표준 Q4 보고서(0.2.10 / 0.1.56, x16 + x4, K=42). 메인테이너가 결과를 웹사이트용으로
  수집하고 0.2.11 새 보고서를 환영한다고 안내해, 이번 보고서를 올립니다.
- 엔진 이슈나 코드 변경은 포함하지 않습니다.

## 접근

- 기존 프로파일 `D:\Strata\launch\ud-q4_k_xl-32k.json` 사용: 32,768 컨텍스트, `--kv int8`,
  `--expert-cache auto`, `--prefill auto`, `--vision`(CPU 인코더), 서버 슬롯 1개.
- `strata bench run --profile ...`가 엔진을 시작하고 모델 로드를 기다린 뒤 표준 스위트를 실행하고, 자기가 띄운
  엔진을 종료했습니다. 폴더의 `engine.log`가 이 실행의 1차 기록입니다.
- 실제 엔진 인자는 **DFlash2**(`--draft dflash2 --spec 6 --draft-conf 0.35`, `--draft-fallback none`)를
  보여 주며, `--layer-split auto`가 **K=25**를 선택했습니다.
- 0.1.57 엔진 패키지가 기존 프로파일의 0.1.56 expert-profile 데이터와 vision 바이너리를 사용했습니다.
  엔진 바이너리와 추론 경로는 수정하지 않았습니다.

패키징:

- 폴더의 `.gitattributes`에 `* -text`를 두어 `SHA256SUMS.txt`가 체크아웃 바이트와 일치하게 했습니다.
- `engine.log`는 저장소가 `*.log`를 무시하므로 `git add -f`로 추가합니다. PCIe probe, K=25, expert cache
  슬롯 수, DFlash2 시작을 확인할 수 있는 유일한 1차 기록입니다.
- `SHA256SUMS.txt`는 폴더의 11개 파일 전체를 나열합니다.

Windows 사용자 참고: 0.2.11 setup은 **새** 다중 AMD 프로파일을 Windows에서 여전히 거부합니다
("several AMD cards sharing one model (--gpus) is Linux-only for now"). 이미 구성된 프로

関連リンク

インストール・モデル・リリースへのサイト内リンク。