Pull requests / #2907

#2907 bench: 2x Arc Pro B50 SYCL IQ3_XXS at 256K context

open · @pankleks · 0 comentarios · En GitHub

BenchmarksMulti-GPUModels & quantsDocumentation

Descripción

## Results-only community report

Two Intel Arc Pro B50 (Battlemage G21, `xe`, 16.2 GB each) with `--layer-split 25`, native SYCL Release JIT build, Strata 0.1.45 at `477ac13c`, original Flash-Next GSQ-RCO **IQ3_XXS** at the full **262,144-token** configured context.

- Official `strata_bench.py`, three runs per case, warm-up and loading excluded, fresh-prompt cache reuse 0.
- Prompt medians: **607.0 / 678.1 / 662.0 tok/s** at actual 4,019 / 16,058 / 32,049 tokens. Pooled short-chat decode **30.8 tok/s** (28.9-40.2, nine capped requests), median TTFT 0.89 s.
- Three six-turn agent/tool sessions: follow-up decode median **33.3 tok/s**, conversation-cache hit median **68.75 %**.
- Needle recall **6/6** at nominal 32K and 128K (depths 10/50/90 %). The tool's 262k rows are refused with HTTP 400 because its 2 %-under sizing collides with a full window; separately, a fresh **258,039-token** prompt completed at **379.5 tok/s** and returned all three anchors, so long prompts are served.
- Documents that prompt throughput falls off with length on this path (662 at 32K, 379 at 258K) while decode does not, and why: KV stays in pinned RAM via `--kv-resident`, so raising the context cost 103 host-served experts and 1 MiB of VRAM.

Bare-metal office host (Ryzen 9 5950X, 60.7 GiB RAM, Ubuntu 26.04.1, oneAPI 2026.1.1), GPU firmware `21.1182`, `STRATA_VERIFY_NO_HOST=0` required by the layer split. README separates the two-GPU split, the shared n-gram shard and the memory plan from what the automatic collector cannot see. Peak VRAM, power and thermals were not captured for this run.

Includes per-request JSON, recall results, hand-collected host/configuration/version/hash facts and reproduction instructions. Credentials, hostnames, office-LAN details and private workload content are excluded. Five new report files only; no engine or headline-performance changes.

En el sitio

Enlaces a install, modelos, releases.