#6477 [Benchmark] Arc Pro B60 / Qwen3.8-Flash-Next / IQ3_S / strata-qwen38-thinking-v1-draft.1

open · @wood3 · 0 comentários · No GitHub

BenchmarksServer & APIMulti-GPUModels & quantsDocumentation

Descrição

## Problem and result

No standard-protocol benchmark exists for a single Intel Arc Pro B60 (24 GB) running Qwen3.8-Flash-Next IQ3_S. The published Intel numbers cover an Arc Pro B70 32 GB, an A750 8 GB, and a 2x Arc Pro B60 layer split; the single-B60 report in the community guide is IQ1_M under the older, non-standard format.

This adds one standard run: **decode 28.1-29.0 tok/s, prompt 654.6-666.4 tok/s at 4K/16K, 12/12 sanity checks passed**, on a card Strata currently reports as untested B-series hardware.

## Related discussion

None. Benchmark reports may go straight to a PR (CONTRIBUTING.md, "Before starting").

## Approach

Results-only change: one new directory `docs/bench-results/2026-11-21-standard-arc-pro-b60-24792a2a/` produced by `strata bench run`. No engine, Terminal, or documentation behaviour changes. The report was not edited by hand; `README.md`, `results.json`, `outputs.jsonl`, `config.json` and the corpus files are exactly what the runner wrote. `provenance/engine.log` is the engine log from the measured server with the user name and host name replaced by `<user>` and `<host>`; no other byte changed. `SHA256SUMS.txt` was computed after that edit.

## Validation

**Passed:**
- `strata bench run --url http://127.0.0.1:8080 --track default --track-notes "Arc Pro B60 24GB, Qwen3.8-Flash-Next IQ3_S, 32k ctx, Intel SYCL engine 0.1.63, 7.5k experts resident"` on the machine below: status complete, 21 of 21 required requests finished, 0 failed, 0 skipped, 5 min 51 s after the model loaded.
- Sanity checks 12 of 12 (instruction 4/4, retrieval 4/4, tool calls 4/4).
- `strata bench report <dir>` re-read the written result without a mismatch.
- Corpus check: the runner verified `bench/prompts/standard-v1` against the bundle manifest before sending (corpus SHA-256 `e75eff8b9feea04f745520b392c4c17d97aefc1ba185e222aa6740efa8f68407`).
- Model file hashes: `00001-of-00002` `4c1eb2ceb4915e11...`, `00002-of-00002` `316b46f3a2dbd68c...` (full values in `results.json`).

**Failed:**
- None observed. No request was retried; zero failed and zero skipped. Five of nine performance requests reached the 512-token cap because the protocol runs with thinking on and `reasoning_effort: xhigh`, so their `total_s` is the time to the cap and not to an answer; the protocol counts these as truncated, not failed.

**Not run:**
- The optional concurrency stage (`--optional-concurrency`) was not requested.
- `drivers` and `runtime` are `not measured` in the report: the runner does not probe the Intel driver version, the Level Zero loader, or the compute runtime. The machine ran Ubuntu 26.04.1 LTS, kernel 7.0.0-44-generic, `xe` driver, `libze1` 1.28.2-2, `intel-opencl-icd` and `libze-intel-gpu1` 26.05.37020.3-1, engine 0.1.63.
- No other GPU, backend, quantization, or context length was measured here.

## Limits

- One card, one model, one quantization, one configuration (32K context, drafting off, pictures off, 3 server slots). Results for a different context, batch size, or expert-cache setting will differ.
- The card is B-series Battlemage, which Strata reports as new and not maintainer-tested; nothing failed in this run, but maintainers may want to treat it as pilot data.
- The machine runs the same kernel and userspace as a desktop, with a GNOME session and the monitor available during the run; no background GPU work was measured or excluded explicitly.
- 7,487 of 24,576 experts were resident in VRAM (14,538 MiB expert cache, 1,775 MiB VRAM free at steady state); the rest were read over PCIe from a 32.64 GiB pinned host mirror. Decode speed depends on that ratio, so a machine with less RAM mirror headroom will be slower.
- The README is time-relative to the local clock of the measuring machine; `results.json` carries the UTC timestamps.

No site

Links install, modelos, releases.