Pull requests / #3633

#3633 [Benchmark] AMD Radeon AI PRO R9700 / Qwen3.8-Flash-Next IQ3_S / strata-qwen38-thinking-v1-draft.1

open · @l33tm4st3r · 0 コメント · GitHub で見る

BenchmarksSetup & installServer & APIAMD / HIPModels & quants

本文

## Problem and result
A standard-protocol run was missing for one Instinct-class consumer card. This PR adds it: one AMD Radeon AI PRO R9700 32 GB (gfx1201), Qwen3.8-Flash-Next GSQ-RCO IQ3_S, protocol ID `strata-qwen38-thinking-v1-draft.1`, status **complete**, 21 of 21 required requests, 94.8 s after the model loaded. Results-only: no engine or tool code changed.

## Approach
One hardware/model/configuration combination, every scheduled outcome retained, required stage and optional concurrency stage kept apart, `configuration.track = custom` (calibrated `--pool-workers 10`, `--vram-reserve-mib 1024` against the 700 setup default, five opt-in `STRATA_*` switches, refusal-direction control vector layers 4-44).

## Validation
Exact commands, on the machine named in the report (R9700 / Ryzen 9 9950X3D / 62 GB RAM / CachyOS kernel 7.3.0-rc6-2-cachyos-rc / ROCm 10.1.0, Strata commit 011c9e60, engine 0.1.47 release binary):

```sh
git checkout 011c9e60
strata profile create iq3_s --from-config strata-iq3_s.json     # -> revision 0414739373b94724
pkill -TERM -f "serve/server.py"                                 # fresh engine, it had answered no request
./pruebas/rocm/run-standard.sh   # = strata bench run --profile iq3_s --load-timeout 2400 --track custom \
                                 #   --track-notes ... --source-revision de4b8e4d43b917e7706784d8bb445c9af86a3540 \
                                 #   --quantization ... --custom-pack ... --background ... --build-options ... \
                                 #   --optional-concurrency --output human
.venv/bin/python strata.py bench report bench/results/2026-10-21-standard-ai-pro-r9700-103ddfcf
```

What I saw.
**Passed:** fixture hashes matched `bench/prompts/standard-v1` before sending; 21/21 required requests finished, 0 failed, 0 skipped; sanity **11 of 12** (instruction 4/4, retrieval 4/4, tool calls 3/4); prompt 2412.8 tok/s (2407.1-2419.3, n=3) at 4k and 3430.3 (3421.4-3444.0, n=3) at 16k; decode 90.4 (89.7-90.6) short; TTFT 0.35 / 1.72 / 4.80 s; optional stage three rounds 79.3 / 86.0 / 88.0 tok/s aggregate at server capacity 1; `strata bench report` re-reads the committed artifacts and reproduces every number.
**Failed:** sanity **T3** - expected one structured tool call, got 0 (kept in the report, not dropped); 6 of 9 performance requests hit the 512-token cap, so they are `truncated` and their totals are time-to-cap.
**Not run / not measured:** 32k and 128k lengths (outside the required suite), concurrency above capacity 1, no baseline pair so no improvement is claimed; `software.drivers`, `hardware.gpus[0].power_limit_w`, `model.revision`, `model.tokenizer_revision` and `protocol.model_card_revision` are null, each with its reason in `results.json` `missing`. Two of them the machine does know, measured separately at the same configuration from the server /metrics at 20:18 and reported only in prose: power limit 330 W (323 W at 48 C) and PCIe gen5 x8 (max gen5, path 4/4).
**Declared, not edited away:** `software.build_kind` says `source` because `strata_cli/bench_facts.py:433` infers it only from `ROOT/.git` existing; the engine here is the release binary setup downloads, so that field is wrong for this run. Code reading is confirmed static analysis, the wrong label is a measured artifact of this report; a `--build-kind` override would fix it (separate issue).

## Limits
Draft protocol, labelled as draft-phase data. One machine, one configuration; the expert cache stays warm across requests, so these are not cold-model measurements. Downloads, compilation, model loading and hashing of the 83.6 GB of shards are excluded from the reported time. The runner carried the unmerged `amd-smi` probe from #3519, which is why `hardware.gpus[0]` names the card instead of the server vendor stub; no engine code was changed.

CI on this head: `check` pass, `title` pass, `sweep` skipping; no Python test job runs because only `bench/results/` changed.

関連リンク

インストール・モデル・リリースへのサイト内リンク。