Pull requests / #2922

#2922 bench: community report, Intel Arc Pro B70 32 GB, GSQ-RCO IQ3_S on engine 0.1.45

open · @kgmkm · 0 commentaires · Sur GitHub

BenchmarksNVIDIA / CUDAModels & quantsDocumentationLinux

Description

## Summary

Community benchmark for engine 0.1.45 on an Intel Arc Pro B70 32 GB
(Ryzen 7 3700X, 121 GB RAM, native Ubuntu 26.04, SYCL AOT build
bmg-g31), serving the official ISTA-DASLab GSQ-RCO IQ3_S at 32,768
context, single-GPU (a B580 drives the display only), with
experimental speed projection enabled. Results only: no engine, build or
test changes.

One run per condition (stated in the README, no median/range), all prompts
unique with 0 reused tokens:

| Configuration | Prompt tok/s | Decode tok/s |
| --- | --- | --- |
| decode-story (48 prompt, 1400 gen) | 56.6 | 27.8 |
| decode-explan (47 prompt, 1299 gen) | 68.8 | 38.0 |
| prompt-4k (4021 prompt) | 421.3 | n/a |
| prompt-20k (19880 prompt) | 1004.5 | n/a |

Draft acceptance 48.9%/51.4%. The README also includes an ESP refusal A/B
on the same prompt: this time both sides generated 500 tokens
(finish=length, no refusal either way), unlike the 0.1.41 RTX 5060 Ti
report where ESP off refused at 106 tokens.

## What changed

- Add `bench/results/2026-10-17-community-arc-pro-b70-iq3_s-0145/`:
the README (hardware, settings, method, tables, limitations),
per-condition medians (`measurements.csv`), one row per request
(`requests.csv`), a `/metrics` snapshot, the sanitized server config and
engine start lines, and the client script.
Local paths are generalized and no credentials are included.

- Add `bench/results/2026-10-14-community-arc-pro-b70-wsl2-iq3_s-0143/`
as a bonus: the same card's older WSL2 GPU-PV numbers (engine 0.1.43.1
plus four uncommitted local WSL2 patches), summary values transcribed
from a 2026-10-14 migration memo — README and `measurements.csv` only,
no per-request data preserved (stated in that README).

- Add nothing else: no code, docs, or index changes (index left to
the maintainers).

- Not attached: the full engine log and the model/pack files. The main
README lists what is missing.

## Extra Notes

- Native Linux on Intel (oneAPI 2026.2, `xe` driver, XFCE desktop on
the B580), so these numbers describe that stack.

- VRAM reserve 500 MiB sits below the engine's own recommendation: a LOW
warning was active (211 MiB free, engine suggests 801), so the main
numbers describe that pressured configuration.

- The story decode (27.8 tok/s) trails the explanation decode
(38.0 tok/s) on lower draft acceptance (48.9% vs 51.4%); outputs
differ, so this is not a controlled decode comparison.

- Not tested: needle recall, TTFT, vision, soak, other quants, larger
contexts, a layer split across the B580, calibration sweeps,
concurrent requests.

Sur le site

Liens install, modèles, releases.