Pull requests / #2056
#2056 hip: gfx103x creates the MTP drafter's side stream again (RX 6800, Windows: 7.9K prompt 584–596 → 798–808 tok/s)
closed · @did-technomancer · 0 comments · View on GitHub
BenchmarksServer & APIMulti-GPUAMD / HIPNVIDIA / CUDAModels & quantsDocumentationWindowsLinux
Description
Fixes the prompt-read regression on gfx103x since 0.1.40.2 (#2055): on those cards the engine creates the MTP
drafter's `side_` stream again. Other HIP cards stay as they are.
## What changes
`MtpDrafter::load` (`src/core/mtp.cpp`): on HIP the `side_` stream is created when the card's architecture starts with
`gfx103`; `STRATA_MTP_SIDE_STREAM=1` / `=0` turns it on or off on any HIP card. The two events (`sh_fork_`, `sh_join_`)
are not created on HIP, as after #1240. CUDA is unchanged. Plus a paragraph in `docs/AMD_HIP.md`.
## Why
#1240 (`ac52c2c7`) dropped the stream and the two events on HIP, because nothing uses them and on an R9700 they slowed
prompts. On an RX 6800 under Windows it is the other way round: the stream's existence alone speeds prompts up, the
events change nothing, and all of the difference is in the `wait copy` phase (the bisect, the 0.1.40–0.1.43 table and
the phase breakdown are in #2055).
## Measurements
```text
GPU AMD Radeon RX 6800 16 GB (gfx1030), two monitors on it
CPU AMD Ryzen 7 5700X3D, 8 cores / 16 threads, AVX2, no AVX-512
RAM 64 GB DDR4-3200 (4x16)
OS Windows 11 Pro for Workstations 26H2 (build 26300)
ROCm 10.2.0a20260930 (TheRock), rocBLAS 5.8.0.efafc1c1
Engine this branch (main 7695859a + this commit), tools\hip\build_windows.bat, STRATA_HIP_ARCHS=gfx1030
Model Qwen3.8-Flash-Next GSQ-RCO IQ3_XXS, --kv int8 --kv-resident 32768, 128K context, MTP --spec 4,
--draft-vocab cyrillic, --prefill 8192; STRATA_HIP_PROMPT_F16=1
```
One binary, one server per row, the arms alternating; the prompts are `serve/server.py`'s code, temperature 0:
| | 7.9K prompt, tok/s | 30.4K prompt, tok/s | 3 prompts of ~850 tokens, tok/s | Decode en / uk / code, tok/s |
| --- | --- | --- | --- | --- |
| `STRATA_MTP_SIDE_STREAM=0` | 584 | 553 | 134–144 | 45.3 / 48.8 / 49.7 |
| unset (the stream exists) | **798** | **863** | 163–168 | 44.3 / 53.8 / 46.0 |
| `STRATA_MTP_SIDE_STREAM=0` | 596 | 600 | 146–164 | 43.0 / 49.5 / 47.3 |
| unset (the stream exists) | **808** | **855** | 158–180 | 44.6 / 50.0 / 50.2 |
The 7.9K prompt reads 35–37% faster and the 30.4K one 43–56% faster. Decode is within the run-to-run spread.
## Not checked
- gfx103x on Linux and with several cards: this is one card under Windows. 2x RX 6900 XT on Linux in #1272 already
showed the opposite sign for `STRATA_SH_STREAM`, so a number from there is needed; `STRATA_MTP_SIDE_STREAM=0` gives
0.1.43's behaviour back.
- R9700 and other HIP cards: nothing changes for them, and I cannot test them.
- The tests (`ctest`) were not run: the build here is `strata` and `strata-device` only.
- Why an unused stream's existence removes `wait copy`: I don't know.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Related on strata.com
Editorial links to help you install, pick models, or read release notes — not part of the upstream thread.