#5908 [Engine] DeepSeek-V4.1 Flash SYCL on B70 — environment and performance

closed · @xiangyuT · 1 Kommentare · Auf GitHub

BenchmarksAMD / HIPModels & quantsDocumentation

Beschreibung

### Area

[Engine]

### What do you want to do?

Add Intel SYCL support for Strata's current DeepSeek-V4.1 Flash engine, including GGUF weights, CPU/GPU experts, Engram and DSpark. 

## Test environment

- **GPU:** one Intel Arc Pro B70, 32 GB.
- **CPU / RAM:** Xeon w5-3435X, 16 cores / 32 threads; host 503.36 GiB. Container limit 192 GiB, 24-CPU quota, 16 engine workers; shared file page cache retained.
- **Software:** Ubuntu 24.04.2, kernel `6.17.0-1007-intel`; `intel/omix:0.4.1-devel-ubuntu24.04`, icpx 2026.1.1, oneMKL 2026.1. Upstream base `13eedf552417a6a791a7d13c6597e24300ee3a09`.
- **Model:** DeepSeek-V4.1-Flash-1017 UD-IQ1_M GGUF, Q8_0 DSpark drafter.
- **Configuration:** context 16384, one slot, top-6, reserve 2048 MiB, mmap on, demand store off, vision off, DSpark on/confidence 0; greedy, thinking off, automatic prefill chunks.

## Performance

Three serial requests per length, after one excluded warmup per length. Median (minimum–maximum):

| Actual input / output | Prefill token/s | Decode token/s | Client TTFT, s | Client total, s |
| --- | ---: | ---: | ---: | ---: |
| 1024 / 256 | 197.0 (191.5–198.4) | 13.0 (12.3–14.5) | 5.203 (5.166–5.350) | 24.921 (22.818–26.190) |
| 4096 / 256 | 377.2 (374.3–378.8) | 12.6 (12.5–13.7) | 10.868 (10.823–10.952) | 31.106 (29.629–31.347) |
| 8192 / 256 | 600.4 (598.8–602.7) | 12.4 (12.2–13.0) | 13.661 (13.609–13.696) | 34.316 (33.346–34.609) |

All nine requests generated exactly 256 tokens, with zero cached prompt tokens and no HTTP/stream errors. Prefill/decode use engine timers; TTFT/total use client timers. Every request reached the output cap. Expert/file caches remained warm. These are absolute custom measurements.

## Work completed

- Add the opt-in SYCL DeepSeek target, GGUF decoding, oneMKL dense operations and CPU/GPU expert integration.
- Implement CPU/GPU synchronization and memory lifetimes; repair portable FP8 rounding and quantized arithmetic.
- Optimize prefill with automatic chunks, dequant-once FP16 expert GEMMs, 1024-row attention slabs and event-based staging/ring ownership.

[Draft PR #5907](https://github.com/Niko1221/Strata/pull/5907) describes the design and validation. [Report, raw results, configuration and reproduction commands](https://github.com/Niko1221/Strata/blob/28952cee2c09dad4df8d60f4541a08d1dc8a3bde/bench/results/2026-11-13-community-b70-deepseek41-sycl-1k4k8k/README.md).

<!-- strata-dsv41-sycl-upstream-20261113 -->

Mehr auf der Site

Links zu Install, Modellen, Releases.