#6314 [Benchmark] RX 6700 XT and RX 6600 (x1, x2, with and without a rocBLAS table) / Qwen3.8-Flash-Next / IQ3_S / strata-qwen38-thinking-v1-draft.1 on the 0.2.16 rx6000 package
open · @Bahise06 · 0 评论 · 去 GitHub 看
BenchmarksMulti-GPUAMD / HIPModels & quantsDocumentationLinux
说明
## Problem and result
Eight standard-protocol runs on the official `strata-engine-0.1.62-linux-x86_64-hip-rx6000` package from v0.2.16, unmodified, on RX 6700 XT 12 GB (gfx1031) and RX 6600 8 GB (gfx1032). `docs/AMD_HIP.md` lists both chips in the package as not tested on a real card. Each card was measured with one and two cards, first without a rocBLAS table, then with a table calibrated on the card. No table existed for gfx1031 or gfx1032, so the engine fell back to rocBLAS's default kernels. The two tables are proposed separately in #6313, so this PR stays results-only. Same machine, model files and switches as #6093 (2x RX 6800 XT), so the three cards compare directly. All eight runs: `Status: complete`, 21 of 21 required requests, 0 failed. Sanity is 12/12 in seven runs and 11/12 in one (see Validation).
| Run | Cards | Table | prompt-4k tok/s | prompt-16k tok/s | decode short / 4k / 16k tok/s | TTFT short s | concurrency tok/s | sanity |
| --- | --- | --- | ---: | ---: | --- | ---: | --- | ---: |
| `b875cff8` | 1x RX 6600 | none | 193.5 | 192.6 | 18.8 / 18.3 / 17.7 | 1.96 | 17.5 / 16.8 / 17.2 | 12/12 |
| `0ac98a70` | 1x RX 6600 | gfx1032 | **225.9** | **240.8** | 18.6 / 17.9 / 18.0 | 1.89 | 17.0 / 17.8 / 17.4 | 12/12 |
| `5e9900e1` | 2x RX 6600 | none | 321.4 | 523.0 | 26.2 / 24.6 / 24.8 | 1.97 | 23.8 / 22.8 / 22.5 | 12/12 |
| `24edcb99` | 2x RX 6600 | gfx1032 | **341.6** | **531.3** | 25.4 / 25.2 / 24.6 | 1.86 | 21.4 / 23.4 / 23.8 | 12/12 |
| `ad926942` | 1x RX 6700 XT | none | 443.6 | 592.6 | 26.1 / 25.0 / 24.8 | 1.59 | 23.7 / 23.4 / 23.5 | 12/12 |
| `ccde3528` | 1x RX 6700 XT | gfx1031 | **468.4** | **599.7** | 25.2 / 24.0 / 25.1 | 1.45 | 23.2 / 24.0 / 23.6 | 11/12 |
| `959d6634` | 2x RX 6700 XT | none | 423.9 | 713.8 | 36.9 / 35.2 / 41.0 | 1.51 | 33.9 / 33.8 / 34.2 | 12/12 |
| `e686f7e1` | 2x RX 6700 XT | gfx1031 | **447.6** | **718.6** | 36.7 / 37.1 / 36.9 | 1.43 | 33.5 / 33.6 / 33.3 | 12/12 |
| #6093 `bc5377e6` | 1x RX 6800 XT (0.2.14) | gfx1030 | 584.7 | 782.0 | 36.2 / 33.3 / 34.4 | 1.25 | 31.3 / 31.7 / 31.8 | 12/12 |
| #6093 `619eb447` | 2x RX 6800 XT (0.2.14) | gfx1030 | 594.7 | 960.6 | 54.1 / 51.5 / 48.5 | 1.10 | 49.4 / 48.2 / 48.1 | 12/12 |
The #6093 rows used the 0.2.14 package and set `STRATA_HIP_PROMPT_F16=1` by hand, with the community gfx1030 table applied. They are context, not a same-version baseline.
- **The table helps one card, and a small one most.** With it, one RX 6600 reads prompts 17% faster at 4K and 25% faster at 16K; one RX 6700 XT gains 6% and 1%. With two cards the gain is 0.7-6%. Decode is unchanged (within run-to-run spread), as expected for a prompt-GEMM table.
- One RX 6700 XT gives about 75-80% of one RX 6800 XT.
- One RX 6600 reads prompts at about 190-240 tok/s whatever the length. With 8 GB the engine picks `prompt chunk auto: 1024 tokens` and an expert cache of 577 slots (1.12 GiB). With two cards the chunk is 6144 tokens and the 16K rate is 2.2-2.7x higher. On the RX 6700 XT the chunk is 8192 tokens with one card or two.
- On 2x RX 6700 XT the 4K prompt rate is slightly below one card (423.9 vs 443.6 without a table, 447.6 vs 468.4 with one). The 16K rate and decode gain (+20%, +41%).
## The two tables (in #6313)
`terminal/tools/hip/gfx1031-rocblas-5.4.0.ad84daa81c.txt` and `gfx1032-rocblas-5.4.0.ad84daa81c.txt` are made with `tools/hip/tune_rocblas` (default shapes and buckets, `--min-gain 1.05`), built against the package's own `libamdhip64.so.7` and `librocblas.so.5`. The headers read `gfx1031` / `gfx1032` and rocBLAS `5.4.0.ad84daa81c`, the build the rx6000 package bundles. A non-default solution won 84 of 112 rows on the RX 6700 XT and 71 of 112 on the RX 6600. The largest single-GEMM gains were on the N=1, K=2560 shape at T=512: 11.2x on gfx1031 and 8.1x on gfx1032. Several larger shapes gained 4-5x (for example N=10240, K=2560 at T=1024 on gfx1031: 4.9x). The tool checked all 896 timed candidates per chip against the default kernel's output and all passed, with a relative L2 of at most 5.0e-5. The engine logged `rocBLAS tuning enabled (112 rows, 84 with a solution, gfx1031, rocBLAS 5.4.0.ad84daa81c)` (gfx1032: 71) in each table run. Each table was calibrated on one card and used for both cards of its pair (same model).
## Related discussion
None needed for a results-only PR. Related: #6093 (the RX 6800 XT runs on this machine), #6094 (CPU prompt share), #5151 (FP16 prompt default), #6154 (gfx1032), #6312 (runner commit lookup), #6313 (the tables).
## Approach
- `strata bench run --profile <json> --track custom --optional-concurrency` from a v0.2.16 checkout (`64524fb9fc`). The track notes give each run's settings. The folders are committed as the runner wrote them.
- Engine options: `--expert-cache auto --prefill auto --spec 4 --mtp <rt head> --pool-workers 5 --max-context 131072 --spec-min-p 0.5 --kv int8 --kv-resident 32768`, plus `--gpu 0` or `--gpu 0,1 --layer-split auto`. Vision off.
- Environment: `HSA_ENABLE_SDMA=0`, `STRATA_HC_PERSIST=1`, `STRATA_PREFILL_CPU_SHARE=auto` with `STRATA_PREFILL_CPU_SHARE_MAX=16384` (#6094), `STRATA_ROCBLAS_TUNING` pointed at a folder.
- In the "none" runs the folder held only the gfx1030 table, so the engine logged `cannot open rocBLAS tuning file (.../gfx1031-rocblas-5.4.0.ad84daa81c.txt); rocBLAS's default kernels run` (gfx1032 likewise).
- In the table runs it held the new tables.
- `STRATA_HIP_PROMPT_F16` unset: every run uses the 0.2.16 default, and the log confirms it on both chips: `HIP device 0 gfx1031` / `gfx1032 - the prompt's 16-bit GEMMs run FP16 in and out by default`.
- Machine: Ryzen 5 5500 (6 cores, PCIe 3.0 only), 62 GB DDR4, Ubuntu 24.04, kernel 6.8. Both cards are on the two CPU PCIe 3.0 x8 slots, the engine probe reads 7.2 GB/s, and the third slot was empty. Resizable BAR off. Headless, with no other GPU process.
## Validation
**Passed:**
- Eight runs complete, 21/21 required requests, 12/12 concurrency requests, 0 failed.
- No engine stall, self-kill or non-finite value in any engine log.
**Failed:**
- `ccde3528` (1x RX 6700 XT, table), sanity T2: "expected one structured tool call, got 0".
- The reasoning says the tool was already called and answers from an imagined result ("We already called... 22°C sunny"), with `finish_reason: stop` after 104 tokens.
- The protocol samples at temperature 1.0, so this is one sample. I cannot attribute it to the table: the two-card run with the same table passed 12/12.
**Not run:**
- Default track.
- KL for the FP16 default or for the tables.
## Limits
- An earlier one-card RX 6700 XT run without a table (`ec9da427`, included) is `nonconforming` only because the runner could not read the Strata commit (#6312). Its numbers match `ad926942`: prompt-4k 442.8, prompt-16k 594.2, decode short 28.2, sanity 12/12. I re-ran it after adding a `terminal/COMMIT` file holding the checkout's HEAD.
- The tables were calibrated once per chip, on one card each.
- "Truncated" means a request reached its 512-token cap, as in the other standard reports.
Prepared with an AI coding assistant. Every number above was measured on this machine.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_018oFQ2YrJ4cibo8xnXRntYH
本站相关内容
相关页面的快捷入口。