#5908 [Engine] DeepSeek-V4.1 Flash SYCL on B70 — environment and performance
closed · @xiangyuT · 1 commentaires · Sur GitHub
BenchmarksAMD / HIPModels & quantsDocumentation
Description
### Area [Engine] ### What do you want to do? Add Intel SYCL support for Strata's current DeepSeek-V4.1 Flash engine, including GGUF weights, CPU/GPU experts, Engram and DSpark. ## Test environment - **GPU:** one Intel Arc Pro B70, 32 GB. - **CPU / RAM:** Xeon w5-3435X, 16 cores / 32 threads; host 503.36 GiB. Container limit 192 GiB, 24-CPU quota, 16 engine workers; shared file page cache retained. - **Software:** Ubuntu 24.04.2, kernel `6.17.0-1007-intel`; `intel/omix:0.4.1-devel-ubuntu24.04`, icpx 2026.1.1, oneMKL 2026.1. Upstream base `13eedf552417a6a791a7d13c6597e24300ee3a09`. - **Model:** DeepSeek-V4.1-Flash-1017 UD-IQ1_M GGUF, Q8_0 DSpark drafter. - **Configuration:** context 16384, one slot, top-6, reserve 2048 MiB, mmap on, demand store off, vision off, DSpark on/confidence 0; greedy, thinking off, automatic prefill chunks. ## Performance Three serial requests per length, after one excluded warmup per length. Median (minimum–maximum): | Actual input / output | Prefill token/s | Decode token/s | Client TTFT, s | Client total, s | | --- | ---: | ---: | ---: | ---: | | 1024 / 256 | 197.0 (191.5–198.4) | 13.0 (12.3–14.5) | 5.203 (5.166–5.350) | 24.921 (22.818–26.190) | | 4096 / 256 | 377.2 (374.3–378.8) | 12.6 (12.5–13.7) | 10.868 (10.823–10.952) | 31.106 (29.629–31.347) | | 8192 / 256 | 600.4 (598.8–602.7) | 12.4 (12.2–13.0) | 13.661 (13.609–13.696) | 34.316 (33.346–34.609) | All nine requests generated exactly 256 tokens, with zero cached prompt tokens and no HTTP/stream errors. Prefill/decode use engine timers; TTFT/total use client timers. Every request reached the output cap. Expert/file caches remained warm. These are absolute custom measurements. ## Work completed - Add the opt-in SYCL DeepSeek target, GGUF decoding, oneMKL dense operations and CPU/GPU expert integration. - Implement CPU/GPU synchronization and memory lifetimes; repair portable FP8 rounding and quantized arithmetic. - Optimize prefill with automatic chunks, dequant-once FP16 expert GEMMs, 1024-row attention slabs and event-based staging/ring ownership. [Draft PR #5907](https://github.com/Niko1221/Strata/pull/5907) describes the design and validation. [Report, raw results, configuration and reproduction commands](https://github.com/Niko1221/Strata/blob/28952cee2c09dad4df8d60f4541a08d1dc8a3bde/bench/results/2026-11-13-community-b70-deepseek41-sycl-1k4k8k/README.md). <!-- strata-dsv41-sycl-upstream-20261113 -->
Sur le site
Liens install, modèles, releases.