#3636 [Benchmark] Radeon AI PRO R9700 + Strix Halo (OCuLink, Linux) / Qwen3.8-Flash-Next / IQ3_S / strata-qwen38-thinking-v1-draft.1

Description

## Problem and result

The 0.2.2 release notes list "Strix Halo with a card of its own on Linux" as not tested ("We have no such machine").
I have that pair (Ryzen AI MAX+ 395, 124 GB, plus a Radeon AI PRO R9700 32 GB on OCuLink PCIe 4.0 x4, Ubuntu,
ROCm TheRock 7.14.1) and ran 0.2.2 (`011c9e6`, source build) on it all day. This PR adds the results only, no engine
changes:

- [`bench/results/2026-10-21-community-strix-halo-r9700-oculink-0.2.2/`](https://github.com/jiayeZhu/Strata/blob/bench-strix-halo-r9700-022/bench/results/2026-10-21-community-strix-halo-r9700-oculink-0.2.2/README.md):
  the two ways to run the pair, measured with fresh engines: the 8060S + R9700 layer split, and the R9700 alone with the
  8060S hidden and the whole expert arena in the Strix Halo's RAM. Long prompts, 1 / 2 / 4 requests at once, short
  prompts, the switches tried (including the ones that failed or did not help), 0.2.1 vs 0.2.2.
- Three standard-v1 runs (draft protocol `strata-qwen38-thinking-v1-draft.1`,
  `strata bench run --url http://127.0.0.1:8080 --suite standard-v1 --optional-concurrency --track custom`, each on a
  fresh engine): the split, the R9700 alone with `--batch 4`, and the R9700 alone one request at a time
  (`STRATA_PREFILL_CPU_SHARE=auto` + `--kv-grow`). All required stages and the optional concurrency stage ran; all
  three are `complete` with 12/12 sanity checks. One at a time on the R9700 is best on short TTFT (0.56 s vs 0.67 s on
  the split), decode (82.9 vs 48.1 tok/s) and the 16K prompt (2,167 vs 1,544 tok/s); the split keeps the 4K prompt
  (1,017 vs 965); `--batch 4` is best when four requests arrive together (24-26 s wall vs 29-33 s on the split).
  - [`2026-10-21-standard-strix-halo-r9700-split-b6c95103/`](https://github.com/jiayeZhu/Strata/blob/bench-strix-halo-r9700-022/bench/results/2026-10-21-standard-strix-halo-r9700-split-b6c95103/README.md)
  - [`2026-10-21-standard-r9700-strix-halo-ram-batch4-c7a2be23/`](https://github.com/jiayeZhu/Strata/blob/bench-strix-halo-r9700-022/bench/results/2026-10-21-standard-r9700-strix-halo-ram-batch4-c7a2be23/README.md)
  - [`2026-10-21-standard-r9700-strix-halo-ram-one-74ba69f0/`](https://github.com/jiayeZhu/Strata/blob/bench-strix-halo-r9700-022/bench/results/2026-10-21-standard-r9700-strix-halo-ram-one-74ba69f0/README.md)

Main findings:

- On this pair the **R9700 alone beats the split**: 64K prompt 1,627 -> 2,530 tok/s, 200K first token 141 s -> 87 s,
  decode at depth 46-49 -> 71-74, four requests at once 82 -> 136-140 tok/s in sum (`--prefill 32768`, arena in RAM).
- Its weak side is short prompts (570 tokens: 186 tok/s against the split's 540): each chunk streams the uncached half
  of the experts over the 7.2 GB/s link. `STRATA_PREFILL_CPU_SHARE=auto` takes it to 429, but only without `--batch`.
- `--kv-grow` on HIP does not start at the default `ulimit -n 1024` (ROCm holds a file per 2 MiB VMM chunk). That is the
  limit #2398 found and raised in the engine; #2398 was closed in the tracker reset, its raise is now the first commit
  of the open #2399, and 0.2.2 does not have it. With the limit raised by hand it works here: 11,697 -> 13,323 experts
  in VRAM (+14%), a 64K prompt +2-3%.
- 0.2.2 = 0.2.1 in speed on the split (within 1%), and the unpatched 0.2.2 split no longer needs the `PF_PAD` patch
  (PR #3484 confirmed on the real pair).

## Related discussion

None needed for a results-only PR. Related: #3484 (PF_PAD, confirmed here); #2706 (gr_upmix on a gfx11 + gfx12 split,
still present in 0.2.2; the split here keeps `STRATA_HC_UPMIX` off); #2398 / #2399 (the open-file limit for
`--kv-grow` on HIP, measured here on 0.2.2); #3519 / #3630 (naming the AMD card in the standard report, see Limits).

## Approach

Every configuration on a freshly started engine, `drop_caches` between them, the machine's other inference containers
stopped, nothing else on either GPU, headless. The community folder follows docs/COMMUNITY_BENCHMARKS.md (README,
per-run JSONL, sanitized configs, the engine logs, the scripts). The standard folders are what `strata bench run` wrote,
plus the sanitized server config, the engine log and a `SHA256SUMS` computed after redaction. Home folder, host name,
LAN addresses and the API key are replaced or removed in every file (so `results.json` differs from the runner's copy
only in those strings); a scan of every file for them found none left.

## Validation

**Passed:**
- standard-v1, three runs: status `complete`, 12/12 sanity checks each (instruction 4/4, retrieval 4/4, tool calls 4/4).
- speed-check: every answer's text right at 1, 2 and 4 requests at once, in every configuration.
- `--kv-grow` on the R9700 alone with the soft open-file limit raised to the hard limit (524288): starts, serves, and
  the K/V takes chunks back as the context grows.

**Failed:**
- `--kv-grow` at the default open-file limit of 1024: the engine never maps the expert cache and does not start
  (logs in the community folder; `scripts/vmm-probe.cpp` stops at the 1,017th chunk with 31.8 GiB free).

**Not run:**
- Anything with a desktop session: this machine is headless, so the screen freezes in the original report are neither
  confirmed nor ruled out. Queue priming on/off (#3480) was not measured.
- #2399's branch (only 0.2.2 with the limit raised by hand).
- Other models and quants, Windows, packaged ROCm (only TheRock 7.14.1), the AppImage packages.

## Limits

One machine, one model (IQ3_S). Short-prompt medians are from 1 to 12 samples per cell. The standard runner reports the
GPU as "AMD Radeon" with no memory, driver or ROCm version (`rocm-smi` and `amd-smi` are installed); with
`ROCR_VISIBLE_DEVICES=0` it is the R9700, on the split both cards are used.

I can keep running things on this pair: if you want a branch, a switch or a model tried on Strix Halo + R9700 under
Linux (the split or the R9700 alone), say so here or mention me in an issue and I will run it and post the numbers.

🤖 Generated with [Claude Code](https://claude.com/claude-code)