#5607 [Benchmark] Radeon 780M (gfx1103): DeepSeek-V4.1-Flash IQ1_M on strata-dsv41, HIPK +80 % prompt, --dspark on +70 % decode
closed · @b4ngcr4nk · 4 Kommentare · Auf GitHub
BenchmarksSetup & installServer & APIMulti-GPUAMD / HIPNVIDIA / CUDAModels & quantsDocumentation
Beschreibung
I got `strata-dsv41` running on a Radeon 780M (gfx1103) and spent a while tuning it. Three switches that are not on by default for this chip made a big difference, so here are the numbers in case they are useful for the 780M docs or the defaults.
Everything below is DeepSeek-V4.1-Flash-1017 IQ1_M (1.75 bpw) at 131,072 context, same machine, measured locally:
| Setting | What I saw |
|---|---|
| `STRATA_DS4_HIPK=1` | prompt 54 -> 98 tok/s at 7K tokens and 68 -> 121 tok/s at 19K (about +80 %), decode unchanged |
| `--dspark on` (with `auto` the drafter stayed off, `draft_n 0`) | decode about 6.5 -> 11 tok/s on code (+70 %), 8.7 on prose (+30 %) |
| `DS4_FAST_ATTN=1` | prompt +16 % (47 -> 55 and 58 -> 68 tok/s) |
No effect that I could measure: `DS4_DQ_OLD=0`, `DS4_ASYNC_T=4` (the Strix Halo docs recommend it), and `--experts 5` (same as `auto` on an integrated GPU, as `experts_per_word()` says).
I tried `STRATA_DS4_HIPK` because the comment in `src/dsv41/ds4_iq_hip.inc` (around line 262) says it is on for gfx12 and gfx1151 and that the other gfx11 parts "were not measured". So this is one data point for that comment.
## Setup
- GMKtec K12: Ryzen 7 H 255 (8C/16T, AVX-512), Radeon 780M (gfx1103), 80 GB DDR5-5600 (48 + 32 GB, dual channel), GTT raised to 44 GiB with `options ttm pages_limit=11534336`
- Proxmox 9.2, kernel 7.0.14-28-pve. The engine runs in an LXC container (Debian 13, 44 GB memory limit) with `/dev/kfd` and `/dev/dri` bound in
- Strata commit `146df3c`, `strata-dsv41 0.1.58 build 146df3c46`, built from source with TheRock ROCm `7.10.0a20251121`
- Model: ISTA GSQ-RCO IQ1_M (121 GB) + Engram IQ4_NL (111 GB) + the DSpark Q8_0 drafter, all on NVMe
Build, the same as in `docs/RADEON_780M.md` plus the dsv41 target (no source changes were needed):
```sh
cmake -S . -B build-dsv41 -G Ninja -DCMAKE_BUILD_TYPE=Release \
-DSTRATA_ENABLE_HIP=ON -DSTRATA_ENABLE_CUDA=OFF -DSTRATA_BUILD_TESTS=OFF -DSTRATA_BUILD_DSV41=ON \
-DSTRATA_PREFILL_MMQ=ON -DSTRATA_Q6K_EXPERTS=ON -DSTRATA_MMQ_KQUANTS=ON \
-DCMAKE_HIP_ARCHITECTURES=gfx1103 <HIP compiler and prefix flags as in RADEON_780M.md>
cmake --build build-dsv41 --target strata-dsv41 -j 12
```
Base engine arguments: `--max-context 131072 --eos 1 --vision off --dspark auto --dspark-conf 0.65 --reserve-mib 700 --experts auto --ssd-cache 24`, with `STRATA_EXPERIMENTAL_GFX1103=1` in the environment. Each row below only changes what the row says.
## How I measured
I restarted the service before every arm (except row 1, which ran on the server that was already up). Prompts are random words (a short list of German nouns, so the token count per word is about 2.5), 2,800 words (about 7.1K tokens) and 7,500 words (about 19K tokens), `temperature 0`, `max_tokens 30`, with a new seed per arm so the prefix cache never helps. Prompt tok/s is `timings.prompt_per_second` from the server. For decode I ran three requests of 150 tokens and one of 300 tokens. The script is at the bottom.
It is **one run per arm and length**. Rows 2 to 6 take the same prompt path (`FAST_ATTN`, no HIPK, the other switches there do not touch the prompt) and landed at 54.0-55.0 and 67.8-68.4 tok/s with different seeds, so I read the run-to-run noise as roughly +-1 %. That is far below the HIPK effect, but it is still not a proper repeat.
| # | Arm | 7K prompt (tokens) | 19K prompt (tokens) | Decode tok/s (150 / 150 / 150 / 300) | Drafter offered/accepted |
|---|---|---|---|---|---|
| 1 | baseline, no switches, drafter off | 47.1 (7066) | 58.5 (19066) | 6.4, 6.5 (30-token probes only) | 0 |
| 2 | `DS4_FAST_ATTN=1` | 54.7 (7121) | 67.9 (19094) | see row 3 | 0 |
| 3 | + `DS4_DQ_OLD=0` | 54.0 (7061) | 68.4 (18968) | 6.3, 6.7, 6.9, 6.7 | 0 |
| 4 | FAST_ATTN + `--experts 5` | 54.7 (7134) | 68.3 (19014) | 6.3, 6.6, 6.8, 6.7 | 0 |
| 5 | FAST_ATTN + `--dspark on` | 54.2 (7041) | 67.8 (18917) | **11.0, 11.9, 11.1, 8.7** | 113/106, 120/111, 120/105, 167/143 |
| 6 | + `DS4_ASYNC_T=4` | 55.0 (7033) | 67.9 (18880) | 11.0, 11.3, 10.5, 8.3 | 112/109, 119/109, 120/102, 151/133 |
| 7 | row 5 + `STRATA_DS4_HIPK=1` | **97.8 (7076)** | **121.5 (18916)** | 11.0, 11.0, 11.0, 8.5 | 116/107, 110/103, 121/105, 172/142 |
| 8 | row 5 + HIPK + `DS4_ASYNC_T=4` | 97.0 (7087) | 120.3 (18961) | 10.5, 11.6, 11.2, 8.1 | 108/105, 117/110, 111/103, 162/130 |
The 30-token probe in row 2 read 6.0 / 5.9 tok/s, which looked like a decode loss. The 150-token runs in rows 3 and 4 show it was just noise.
A greedy answer to "count from 1 to 40" had the same md5 (`6af0d73e`) in all eight arms. That is a single short probe. **I did not measure KL or perplexity against a reference**, so I can't say HIPK is lossless. It does change the operand type (F16 WMMA instead of F32 FMA).
Some other things I saw:
- Long prompts (measured before HIPK, with FAST_ATTN): a 32,140-token prompt took 529 s (61.5 tok/s) and a 70,795-token prompt, of which 33,288 tokens were new because of the prefix cache, took 563 s (59.8 tok/s). Both finished at 131K context, decode 5.8-5.9 tok/s. I have not repeated long prompts with HIPK.
- During prompt processing the iGPU is 96 % busy at 2.3-2.6 GHz and 54 C, CPU around 80 C, no throttling that I could see, and no amdgpu reset or fault in `dmesg` during the HIPK runs.
- Outside of Strata I run a small 20-task agent test (read, search, edit, run tests over the OpenAI API, checked automatically). With HIPK on it finished 20/20 with 0 of 96 tool calls malformed, in 13.9 min (21.5 min with none of the switches). One task ("write a file") runs into its 14-step limit in roughly half of the runs in every arm I tried (HIPK and drafter on 4/8, HIPK off 3/8, drafter off 4/8), so I put that on the model and not on a switch. It always gets solved.
## Not done
- `strata bench run` / the standard protocol
- needle or recall tests, or any accuracy comparison for DeepSeek on this GPU
- a `DS4_PROF` / `DS4_FLOW_STATS` breakdown
- long prompts with HIPK, other gfx11 chips, and more than one run per arm and length
## Question
Would you want `STRATA_DS4_HIPK` enabled by default for gfx1103, or just mentioned in `docs/RADEON_780M.md` as a measured opt-in for the DeepSeek engine (together with `--dspark on`, since `auto` left the drafter off here)? If you would rather have more data first (repeats, a KL comparison, 32K prompts) or a docs-only PR, tell me which and I will do that before anyone touches a default.
<details>
<summary>Measurement script</summary>
```python
import json,urllib.request,time,random,hashlib,sys
label=sys.argv[1]
W="Datenbank Cache Kernel Thread Speicher Netzwerk Protokoll Treiber Scheduler Puffer Index Query Transaktion Lock Seite Block Pipeline Shader Queue".split()
def ask(p,mt):
b={"model":"deepseek-v4.1-flash","max_tokens":mt,"temperature":0,"messages":[{"role":"user","content":p}],"chat_template_kwargs":{"enable_thinking":False}}
r=urllib.request.Request("http://127.0.0.1:8080/v1/chat/completions",json.dumps(b).encode(),{"Content-Type":"application/json"})
t0=time.time();d=json.load(urllib.request.urlopen(r,timeout=3600));return d,time.time()-t0
print("==",label,flush=True)
d,_=ask("Zaehle die Zahlen von 1 bis 40 auf, getrennt durch Komma. Nur die Zahlen.",120)
c=d["choices"][0]["message"]["content"]; print("probe",hashlib.md5(c.encode()).hexdigest()[:8],repr(c[:60]),flush=True)
for seed,words in ((int(sys.argv[2]),2800),(int(sys.argv[3]),7500)):
random.seed(seed)
txt=" ".join(random.choice(W)+str(random.randint(0,999)) for _ in range(words))
d,s=ask(txt+"\n\nFasse in einem Satz zusammen, worum es geht.",30); t=d["timings"]
print(f'prefill {t["prompt_per_second"]:.1f} tok/s ({t["prompt_n"]} Tok, {s:.0f}s) | decode {t["predicted_per_second"]:.2f} tok/s',flush=True)
```
Run as `python3 pf.py LABEL SEED_7K SEED_19K` against the running server.
</details>
Mehr auf der Site
Links zu Install, Modellen, Releases.