Pull requests / #2922
#2922 bench: community report, Intel Arc Pro B70 32 GB, GSQ-RCO IQ3_S on engine 0.1.45
open · @kgmkm · 0 comentarios · En GitHub
BenchmarksNVIDIA / CUDAModels & quantsDocumentationLinux
Descripción
## Summary Community benchmark for engine 0.1.45 on an Intel Arc Pro B70 32 GB (Ryzen 7 3700X, 121 GB RAM, native Ubuntu 26.04, SYCL AOT build bmg-g31), serving the official ISTA-DASLab GSQ-RCO IQ3_S at 32,768 context, single-GPU (a B580 drives the display only), with experimental speed projection enabled. Results only: no engine, build or test changes. One run per condition (stated in the README, no median/range), all prompts unique with 0 reused tokens: | Configuration | Prompt tok/s | Decode tok/s | | --- | --- | --- | | decode-story (48 prompt, 1400 gen) | 56.6 | 27.8 | | decode-explan (47 prompt, 1299 gen) | 68.8 | 38.0 | | prompt-4k (4021 prompt) | 421.3 | n/a | | prompt-20k (19880 prompt) | 1004.5 | n/a | Draft acceptance 48.9%/51.4%. The README also includes an ESP refusal A/B on the same prompt: this time both sides generated 500 tokens (finish=length, no refusal either way), unlike the 0.1.41 RTX 5060 Ti report where ESP off refused at 106 tokens. ## What changed - Add `bench/results/2026-10-17-community-arc-pro-b70-iq3_s-0145/`: the README (hardware, settings, method, tables, limitations), per-condition medians (`measurements.csv`), one row per request (`requests.csv`), a `/metrics` snapshot, the sanitized server config and engine start lines, and the client script. Local paths are generalized and no credentials are included. - Add `bench/results/2026-10-14-community-arc-pro-b70-wsl2-iq3_s-0143/` as a bonus: the same card's older WSL2 GPU-PV numbers (engine 0.1.43.1 plus four uncommitted local WSL2 patches), summary values transcribed from a 2026-10-14 migration memo — README and `measurements.csv` only, no per-request data preserved (stated in that README). - Add nothing else: no code, docs, or index changes (index left to the maintainers). - Not attached: the full engine log and the model/pack files. The main README lists what is missing. ## Extra Notes - Native Linux on Intel (oneAPI 2026.2, `xe` driver, XFCE desktop on the B580), so these numbers describe that stack. - VRAM reserve 500 MiB sits below the engine's own recommendation: a LOW warning was active (211 MiB free, engine suggests 801), so the main numbers describe that pressured configuration. - The story decode (27.8 tok/s) trails the explanation decode (38.0 tok/s) on lower draft acceptance (48.9% vs 51.4%); outputs differ, so this is not a controlled decode comparison. - Not tested: needle recall, TTFT, vision, soak, other quants, larger contexts, a layer split across the B580, calibration sweeps, concurrent requests.
En el sitio
Enlaces a install, modelos, releases.