Pull requests / #3581
#3581 [Benchmark] Radeon VII 16 GB (gfx906) - standard-v1 (complete) + --pool-workers sweep (release v0.2.1.4, engine 0.1.46.3)
open · @Kaidora · 0 comments · View on GitHub
BenchmarksServer & APIMulti-GPUAMD / HIPModels & quantsDocumentation
Description
## Problem and result A complete run of the draft standard protocol on one AMD Radeon VII 16 GB (gfx906), Qwen3.8-Flash-Next Q2_0 with MTP, 65536-token context, release v0.2.1.4 (engine 0.1.46.3) built from source for gfx906 with no local patches. `strata bench run --suite standard-v1`, started from a freshly loaded engine (the report notes no earlier requests), status **complete**: 21 of 21 required requests, 0 failed, 0 skipped, **12 of 12 sanity checks passed** (instruction 4/4, retrieval 4/4, tool calls 4/4). Time after the model loaded: 5 min 37 s. | case | prompt tok/s | decode tok/s | first token | total | | --- | ---: | ---: | ---: | ---: | | short | 65.1 (65.0-67.9) | 50.0 (43.9-53.0) | 1.22 s | 9.6 s | | prompt-4k | 484.5 (483.6-485.8) | 45.7 (44.5-51.0) | 8.49 s | 19.6 s | | prompt-16k | 524.2 (514.2-527.1) | 47.8 (45.5-49.9) | 31.32 s | 42.5 s | The folder also carries a ten-run supplementary sweep of the community guide's quick suite (`pool-workers-sweep/`). It is attached because the protocol fixes its prompts, thinking and sampling for cross-machine comparison, while the one open question on this machine needs a single setting varied: the engine's default for `--pool-workers` is 15 workers on this 16-core CPU, and 10 or 6 decode 11-23 % faster (three repeats of the default arm at each context put the effect at 3.8-5.7x the replicate spread at 65536 and 1.9-2.2x at 262144, permutation p <= 0.0016). Those ten runs are **not** standard-protocol results and the report says so. ## Related discussion none ## Approach Results-only: one new folder under `bench/results/` with the standard run's `README.md`, `results.json`, `outputs.jsonl`, `config.json`, `provenance/` (chat template, corpus build and manifest), plus `pool-workers-sweep/` with ten `results.json` files and its own README. No engine code changes. The protocol is a draft (`strata-qwen38-thinking-v1-draft.1`); this report records its draft status and does not claim standard or approved status. The run is `track: custom` because the configuration is tuned (`--expert-cache auto`, `--kv int8 --kv-resident 32768`, `--spec 4 --spec-min-p 0.5`, a local MTP directory, `--pool-workers 6`). ## Validation **Passed:** - Build: `cmake -S <checkout at tag v0.2.1.4 (engine 0.1.46.3)> -B build-906 -G Ninja -DSTRATA_HIP_GFX906=ON -DCMAKE_HIP_ARCHITECTURES=gfx906 -DSTRATA_GGML_DIR=<llama.cpp at 3cf03257> -DCMAKE_C_COMPILER=/opt/rocm/llvm/bin/clang -DCMAKE_CXX_COMPILER=/opt/rocm/llvm/bin/clang++` then `ninja -C build-906 strata`: 158 of 158 steps, no patches. - Load and serve: `strata --serve ... --max-context 65536` answers `/health` with `max_context: 65536`; the standard run started from that fresh engine and reported no earlier requests. - `strata bench run --suite standard-v1`: status complete, 21/21 requests, 0 failed, 0 skipped, 12/12 sanity checks. - The ten quick-suite runs: 0 failed requests. **Failed:** - none observed. The standard run excludes one request from its speed summaries because a first-answer timestamp was not available (`performance-prompt-4k-r3`); the report records it. **Not run:** - pictures (`--vision`: no `mmproj` on this card), `--clients`, `--long`, a second build for comparison, a long soak. - A standard-protocol run at 262144 context: the protocol's cases are 4K and 16K, and this card's 262144 context streams the KV, so only 65536 is covered by the protocol run; the 262144 numbers in the sweep are quick-suite numbers. - No conformance claim beyond the runner's own `complete` status. ## Limits - One card, one CPU, one machine; nothing here is a claim about other gfx906 cards. - The protocol is a draft, and its track is `custom`, so this is not a headline "standard" number. - The quick-suite sweep uses `temperature: 0` with thinking off and no sanity checks; it isolates one setting and is not comparable with the protocol's numbers. - The two contexts in the sweep are not comparable with each other (65536 keeps the KV in VRAM, 262144 streams it). - The agent test in the sweep carries more run-to-run noise than its chat test.
Related on this site
Quick links to related pages.