Pull requests / #2907
#2907 bench: 2x Arc Pro B50 SYCL IQ3_XXS at 256K context
open · @pankleks · 0 コメント · GitHub で見る
BenchmarksMulti-GPUModels & quantsDocumentation
本文
## Results-only community report Two Intel Arc Pro B50 (Battlemage G21, `xe`, 16.2 GB each) with `--layer-split 25`, native SYCL Release JIT build, Strata 0.1.45 at `477ac13c`, original Flash-Next GSQ-RCO **IQ3_XXS** at the full **262,144-token** configured context. - Official `strata_bench.py`, three runs per case, warm-up and loading excluded, fresh-prompt cache reuse 0. - Prompt medians: **607.0 / 678.1 / 662.0 tok/s** at actual 4,019 / 16,058 / 32,049 tokens. Pooled short-chat decode **30.8 tok/s** (28.9-40.2, nine capped requests), median TTFT 0.89 s. - Three six-turn agent/tool sessions: follow-up decode median **33.3 tok/s**, conversation-cache hit median **68.75 %**. - Needle recall **6/6** at nominal 32K and 128K (depths 10/50/90 %). The tool's 262k rows are refused with HTTP 400 because its 2 %-under sizing collides with a full window; separately, a fresh **258,039-token** prompt completed at **379.5 tok/s** and returned all three anchors, so long prompts are served. - Documents that prompt throughput falls off with length on this path (662 at 32K, 379 at 258K) while decode does not, and why: KV stays in pinned RAM via `--kv-resident`, so raising the context cost 103 host-served experts and 1 MiB of VRAM. Bare-metal office host (Ryzen 9 5950X, 60.7 GiB RAM, Ubuntu 26.04.1, oneAPI 2026.1.1), GPU firmware `21.1182`, `STRATA_VERIFY_NO_HOST=0` required by the layer split. README separates the two-GPU split, the shared n-gram shard and the memory plan from what the automatic collector cannot see. Peak VRAM, power and thermals were not captured for this run. Includes per-request JSON, recall results, hand-collected host/configuration/version/hash facts and reproduction instructions. Credentials, hostnames, office-LAN details and private workload content are excluded. Five new report files only; no engine or headline-performance changes.
関連リンク
インストール・モデル・リリースへの站内リンク。