Pull requests / #2114
#2114 sycl: run on an integrated Arc (Arc 140T, xe): free memory from /proc/meminfo, no prompt-slot borrowing, a native launcher
open · @andersson82 · 0 Kommentare · Auf GitHub
BenchmarksSetup & installServer & APIAMD / HIPNVIDIA / CUDAModels & quantsDocumentationWindowsLinux
Beschreibung
## Title sycl: run on an integrated Arc (Arc 140T, xe): free memory from /proc/meminfo, no prompt-slot borrowing, a native launcher Issue: none; the "integrated Arc GPUs ... not tested" line of docs/INTEL_ARC.md (and #515 for the B390 on Windows, not covered here) ## Summary The SYCL port did not start on an integrated Arc: the Arc 140T has no free-memory query, so the expert cache was sized from 0. With the three changes below it runs the Coder IQ1_M on the 140T with 62 GiB of RAM, through the normal server, at 12.0-17.4 tok/s decode, and reads a 72,648-token prompt at 156.5 tok/s. ## What changed - `sycl/include/dpct/device.hpp`, `get_memory_info`: on a device with `host_unified_memory` and no `ext_intel_free_memory`, free memory is `MemAvailable` from `/proc/meminfo` (Linux only); `STRATA_SYCL_FREE_MIB` sets it by hand (the `xe` driver keeps freed GPU memory in a pool that `MemAvailable` does not count until a reboot). Other devices: the code path is the one before. - `sycl/src/program/generate.cpp`: no prompt-slot borrowing on a `host_unified_memory` device. With borrowing, the answer after a borrowing prompt was token 0 (every expert activation NaN from the second prompt chunk; measured on the 0.1.40 port, any context). Discrete cards keep the old rule. - `sycl/serve/strata-sycl-native.sh` (new): the config's `exe` for the engine on the PC's own oneAPI, no Docker. Sets the same port defaults as `strata-sycl.sh` (each overridable from the config's `"env"`) plus `STRATA_NO_MULTI_GR=1` (15.1 vs 18.3 tok/s on the 140T with the multi-token GR kernels, 0.1.40, same output) and an OOM score of 1000 for the engine. - `docs/INTEL.md`: a new section "Integrated Arc: the Arc 140T" (build, what is different from a card, a config to run it by hand, the numbers). `docs/INTEL_ARC.md`: the 140T is no longer in "not tested". Left alone: discrete cards (every change is behind `host_unified_memory`, which a card with its own VRAM should not report; I have no discrete card to check), CUDA and HIP (no file of theirs is touched), setup (its own PR, which builds on this one). ## Measurements Arc 140T (8086:7d51, Xe-LPG+), Core Ultra 9 285H, 62 GiB RAM, Fedora 44 (kernel 7.2.9, `xe`), oneAPI 2026.0, JIT build. Coder IQ1_M, `--spec 4 --mtp`, `--max-context 98304`, `--kv int8`, `--stream-experts`, through `sycl/serve/server_intel.py`, greedy, thinking off. Engine 0.1.43 with these changes and #2110 (without #2110 every request stalls on the 140T). | | | |---|---| | decode, 100-token answers, 10 requests | 12.0-17.4 tok/s, all 12,288 experts cached (first start after a boot) | | the same, later start (the `xe` pool) | 11.5-17.2 tok/s, 10,496 cached, 1,792 mirrored | | a 72,648-token prompt | read at 156.5 tok/s, the fact on its first line recalled | | first start / later starts | 79 s / 7 s with `SYCL_CACHE_PERSISTENT=1` (0.1.40) | ## Extra Notes How I checked: - Built this branch alone on `origin/main` (7695859a) with `sycl/tools/build.sh` on the PC's oneAPI: 0 errors. - `bash -n` on the launcher. Not tested: - Any discrete Intel card (I have none); the code paths there should be unchanged. - Other integrated GPUs (Lunar Lake, Panther Lake), `i915` on an iGPU, Windows. - The general model at 0.1.43 (IQ2_XS ran on the 0.1.40 port: 13.3-16.2 tok/s decode). - Where a decode round's time goes (not profiled). 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Mehr auf der Site
Links zu Install, Modellen, Releases.