Pull requests / #2079
#2079 Add native Windows support for Intel SYCL GPUs
open · @lm902 · 0 commentaires · Sur GitHub
BenchmarksSetup & installAMD / HIPNVIDIA / CUDAModels & quantsDocumentationWindowsLinux
Description
## Summary Add native Windows support for the Intel SYCL backend, using the existing START-HERE.bat installer entry point. This PR contains only the necessary Windows/Intel compatibility code and installer regression tests. It does not contain models, local configurations, benchmark scripts, logs, dumps, documentation changes, or user backups. This is a new, independent PR based on upstream main at `7695859aa13afb248bdb4d36f9b2e95707db079d` (engine 0.1.43). It does not reuse or modify the earlier conflicted PR. ### Changes - Route an Intel-only Windows installation to native SYCL; retain the existing NVIDIA/AMD default on mixed-GPU systems and honor an explicit backend choice. - Build with Visual Studio C++ and Intel oneAPI using `sycl/tools/build_windows.bat`; use Windows floating-point/MKL flags and required Windows libraries. Keep Linux flags and paths unchanged. - Discover the Intel GPU and 64-bit VRAM size before an engine exists, then use the native engine's JSON device probe. Report PCI device IDs so A-series FP64 emulation settings are retained. Pin a selected Windows GPU through ONEAPI_DEVICE_SELECTOR instead of relying on CUDA_VISIBLE_DEVICES. - Use native paths, runtime DLL search directories, the real Windows RAM budget, and the existing Intel server wrapper. Keep GPU device-plan/no-host safeguards enabled on the native streamed path. - Replace POSIX GGUF positional reads with independent Win32 overlapped reads and pointer-width file handles. Query available physical RAM with GlobalMemoryStatusEx for the expert-mirror budget. - Do not label an ordinary Windows VirtualAlloc arena as registered GPU host memory. Enable the Level Zero host-allocation path only when both its headers and loader library are available. - Resolve Windows SDK macro collisions and replace parity-test environment setters with their Windows equivalents. No quantization formats or mathematical kernel algorithms are changed. ## Test environment Measured locally on October 11-12, 2026, using the upstream baseline above plus this Windows port: | Component | Environment | | --- | --- | | OS | Windows 11 Pro for Workstations Insider Preview, build 26220, x64 | | GPU | Intel Arc Pro B70, nominal 32 GB; runtime reports 31.159 GiB | | Intel graphics driver | 32.0.101.9034 | | CPU | Intel Core i9-13900KF; AVX2, no AVX-512 | | RAM | 64 GB installed | | Build | Visual Studio 2022 17.14.40, MSVC 14.44; Windows SDK 10.0.26100 | | Intel toolchain | oneAPI DPC++/C++ 2026.1.1; oneMKL 2026.1.0 | | Engine | Native x64 SYCL/Level Zero, Release/Ninja, SPIR-V JIT; stateless >4-GiB addressing | | Python | 3.11.9 | No Docker, WSL, or /dev/dxg was used. Tests explicitly selected `level_zero:gpu`, disabled the persistent SYCL cache, and retained `STRATA_VERIFY_DEVICE_PLAN=1` and `STRATA_VERIFY_NO_HOST=1`. Disabling persistent caching is a recorded test setting, not a new installer default. CUDA/HIP hardware and Linux execution were not validated on this machine. ## Build and regression validation Re-run after the final Windows device-selection/PCI-ID fixes: - `python -m unittest discover -s tools -p "test_setup*.py"`: **479 tests passed**. - `sycl\tools\build_windows.bat`: **all Windows SYCL targets built successfully**. - Native `strata.exe --list-devices`: correctly reports the B70, 31.159 GiB, Level Zero, PCI device ID `e223`. - `ctest --test-dir build-sycl-win --output-on-failure -E "^(gr_parity|ple_parity|s2_expert_grouped_parity)$"`: **30/30 passed** on the B70. This is not a claim that all 33 tests passed. The exclusions encountered earlier were `gr_parity` (unsupported QFUSE path), `ple_parity` (missing external fixture), and `s2_expert_grouped_parity` (bitwise mismatch); they are not changed or hidden by this PR. - The original START-HERE.bat path, model-menu installation, native build/start, and real chat were tested during port validation. Earlier full-model checks passed all 48 expert layers and the resident-cache byte check (0/12288 mismatches). - `git diff --check`: passed. ## Local performance measurements These are functional/capacity measurements, **not before/after speedup claims**, quality benchmarks, or a general hardware recommendation. Each reported case is one warmed measurement, not a multi-run average. Inputs were identical within each model-comparison test; first-token changes prevented prefix-cache reuse. Successful cases had zero reused prompt tokens and generated exactly 256 new tokens. Expert computation stayed on the GPU, including experts read over PCIe from pinned RAM. The total model size must not be interpreted as all weights fitting in VRAM; the PLE table stays on disk. ### Original models and Coder: preallocated 262144-token capacity Protocol: native `--kv int8` (64 values per FP16 scale, **not strict GGML Q8_0**), `STRATA_KV_GROW=0`, auto expert cache, streamed experts, 2048 MiB VRAM reserve, prefill chunks of 4096, 15 pool workers, 12 stager threads, temperature 0, MTP maximum 4 and `--spec-min-p 0.5`. The engine also has lookup drafts; the aggregate INFO spec count is 6, so this is not non-speculative decoding. A 4096-input/128-output warmup preceded each run. The 255K tests used **261120 actual input tokens**, leaving room within 262144 for 256 output tokens. KV modes: **GPU** = all allocated KV in VRAM (`--kv-resident 0`); **stream** = full KV in RAM plus a 32768-token VRAM window (`--kv-resident 32768`). Values below are **prefill / decode tokens per second**. | Model | KV mode | 128 input | 32768 input | 131072 input | 261120 input | | --- | --- | ---: | ---: | ---: | ---: | | Coder IQ1_M | GPU | 180.9 / 44.4 | 1339.1 / 46.7 | 1143.8 / 41.6 | 964.6 / 40.9 | | Q2_0 | GPU | 179.5 / 54.3 | 992.2 / 55.4 | 1067.8 / 50.8 | 888.4 / 51.3 | | Q2_0 | stream | 210.1 / 65.6 | 1273.1 / 60.3 | 1103.4 / 54.0 | 916.0 / 54.8 | | IQ2_XS | GPU | 180.1 / 61.6 | 1177.4 / 61.4 | 1018.1 / 57.6 | 860.7 / 54.8 | | IQ2_XS | stream | 189.5 / 76.1 | 1200.6 / 71.8 | 1039.9 / 66.8 | 874.8 / 64.8 | | IQ3_XXS | GPU | 141.8 / 40.2 | 1061.2 / 42.6 | 936.7 / 40.4 | 794.8 / 38.7 | | IQ3_XXS | stream | 150.6 / 47.2 | 1086.0 / 49.4 | 955.0 / 47.2 | 807.5 / 45.9 | | IQ3_S | GPU | 109.9 / 28.0 | 968.1 / 26.6 | 863.6 / 32.4 | 743.0 / 32.1 | | IQ3_S | stream | 125.2 / 32.7 | 996.8 / 31.1 | 891.3 / 37.7 | 760.4 / 37.1 | Original-model repository revision: `ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF` at `ed59f92082b1e93c0e96d60a8b11aab089b52f09`. Total GGUF sizes (decimal GB): Coder IQ1_M 58.409; Q2_0 66.424; IQ2_XS 68.026; IQ3_XXS 75.840; IQ3_S 83.618. IQ3_S was the largest original tier actually tested successfully, not a proven absolute capacity limit. At 261120 input tokens, IQ3_S sampled dedicated GPU memory peaked at approximately 28.48 GiB in both modes; shared GPU memory peaked at 30.31 / 31.05 GiB, engine working set at 32.58 / 33.41 GiB, and minimum available system RAM at 7.73 / 7.15 GiB (GPU / stream). Shared GPU memory overlaps the process working set and must not be added to it; process private commit is not physical RAM usage. Successful paired original-model outputs matched, including the 255K tests. ### OrcaRouter IQ3_XXS: independent compatibility-pack check Local two-shard `orcarouter/Qwen3.8-Flash-Next-Uncensored-GGUF`, 85.203 GB total, download revision `d27c0a15447dd92ae7c90919352c6810496945b1`; both SHA256 hashes matched the local download metadata. Prepared an independent `--compat-bf16` pack and used this model's own tokenizer/chat template. Converted 459 tensors (96 exact, 363 rounded); 1.22 GiB BF16 conversions and 1.38 GiB packed dense arena. Expert bytes and PLE table were unchanged. This conversion does not recover the original BF16 checkpoint or prove its quality. Protocol differs from the table above: **32768 capacity, 512-token prefill chunks, full-GPU INT8 KV**, original expert profile and original MTP runtime as in docs/ORCA.md, MTP maximum 4, temperature 0, 2048 MiB reserve, native streamed experts/no CPU fallback. Arithmetic returned 2603 for 137*19; translation returned "你好,世界!"; the generated square(x) function passed an AST syntax check (generated code was not executed). | Actual input tokens | Prefill tok/s | Prefill seconds | Decode tok/s | Decode seconds | | ---: | ---: | ---: | ---: | ---: | | 128 | 33.7 | 3.80 | 11.5 | 22.26 | | 8192 | 140.4 | 58.34 | 20.5 | 12.48 | | 31744 | 278.9 | 113.80 | 22.5 | 11.39 | All three cases generated 256 tokens, finished at the output-length limit, reused no prefix cache, and did not trigger the RAM guard. GPU log: 11193 resident experts (22.68 GiB), all 13383 remaining experts mirrored (27.12 GiB), resident experts computed on the B70. Sampled peaks: dedicated GPU 28.35 GiB, shared GPU 27.54 GiB, engine working set 29.16 GiB; minimum available system RAM 15.51 GiB. This used the Windows GPU-cache/RAM-mirror path rather than the document's full approximately 49.8-GiB expert arena. HTTP/API, vision, multi-turn/cancellation and 256K Orca operation were not tested in this check. ## Failures and limits disclosed - **Coder with streamed KV was not stable.** An initial short/32K run completed, but 128K prefill hit the 60-second stall watchdog. A same-parameter retry stalled during 32K. No RAM guard triggered; the cause is not established. It is intentionally omitted from the successful-performance table and was not tested at 255K. - **Unsloth UD-IQ4_XS and UD-Q4_K_XL did not complete benchmarks** on this 32-GB GPU/64-GB RAM machine with 262144 preallocated INT8 KV. At 4096-token chunks, the no-host coverage guard rejected both KV modes: IQ4_XS had 377/163 uncovered experts; Q4_K_XL had 5574/5572 (GPU/stream). - An isolated **IQ4_XS 2048-token chunk retry** reached READY in both modes, but the 4096-token warmup was canceled after 2048 input tokens with zero output: available RAM fell to 1.54/1.84 GiB, below the 2-GiB test guard. Those canceled requests have **no valid performance result**. No guard was disabled or context capacity reduced to force success. This is a tested configuration limit, not a universal statement that the formats cannot run. - Performance varies with prompt, draft acceptance, current desktop VRAM use, and I/O. The Orca 512-chunk protocol is not directly comparable with the original-model 4096-chunk protocol or the document's RTX 5090/Linux example. All local models, configs, raw measurements and test artifacts are preserved outside the commit. No existing PR has been edited or closed.
Sur le site
Liens install, modèles, releases.