#6448 [Engine] Scope: plan an integrated Intel GPU (Arc B390, Panther Lake) as one pool of memory with the system in the SYCL engine
open · @criskurtin · 0 commentaires · Sur GitHub
BenchmarksSetup & installServer & APIModels & quantsSecurityDocumentationWindows
Description
### Area [Engine] (speed, kernels, models, the server and API) ### What do you want to do? Run the SYCL engine on an Intel GPU built into the processor, with its memory planned as what it is: one pool shared with the system. My machine is an Arc Pro B390 (Panther Lake, `8086:b086`, `xe`) with 62 GiB of LPDDR5X shared with the GPU, on Fedora 44 (kernel 7.2.11, compute runtime 26.35.39758). I can be the integrated-Arc tester that cef9bca88 says is missing; `docs/INTEL_ARC.md` still says "shared-memory planning does not exist". **On main 3142b66b9,** dpct never calls `set_integrated()` (bug 1 of #2282, fixed by #3729), so #409's MemAvailable rule never runs. The expert cache is sized from Level Zero's free figure, which follows the page cache, and the experts that do not fit go to a pinned host mirror in the same RAM. With #3729 the cache is sized from MemAvailable less 6 GiB, but a mirror is still built. After an idle unload, `xe` keeps the freed pages pooled (`GPUReclaim` in `/proc/meminfo`, 15-29 GiB here; #2412's "TTM page pool"). MemAvailable does not count them, so a second start sizes a smaller cache than the first. **Proposed,** only on a device that reports itself as integrated (discrete cards keep main's path): 1. Detection: #3729 as it is, with its author. 2. Sizing: MemAvailable + `GPUReclaim` − `STRATA_UMA_HEADROOM_GIB` (default 6, unchanged), capped at the device's global memory. A kernel without the `GPUReclaim` line, or one that reports 0 (7.0 in #2412), adds nothing. 3. No second copy in the same RAM: an auto-sized cache gets no pinned mirror (experts that do not fit are read from the file), the prompt loan's lend mirror is off, and KV streaming is not turned on by itself. With `STRATA_VERIFY_NO_HOST`, a model that does not fit is refused as soon as the cache is sized, with the shortfall in GiB, instead of after the fill. 4. Separately: Xe3 classed with Battlemage in `intel_gpu_gen()`, with a `STRATA_INTEL_GEN` override. Runtime 26.35 names this GPU "Intel(R) Graphics [0xb086]" with architecture `intel_gpu_ptl_h`, so main gives it the generic kernels. Measured with the Swift IQ3_XXS files of [`huihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated-v2-GGUF`](https://huggingface.co/huihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated-v2-GGUF) (Swift 1.5's two-file layout and quant types; 24,576 experts, 40 GiB), `terminal/tools/ab_engine.py`, 3 rounds, arms alternated, a fresh server per arm. Run config: `--expert-cache auto --stream-experts --prefill auto --prefill-borrow --spec 4 --spec-min-p 0.5 --mtp <rt> --max-context 131072 --kv int8 --vram-reserve-mib 1024`, no `STRATA_*` variables. | build | expert cache at start | pinned mirror | lowest MemAvailable | decode tok/s, code prompt (median of 3) | |---|---|---|---|---| | main | 8,425-13,400 slots | 19.2-25.4 GiB | 4.4-8.7 GiB | 19.1 | | main + #3729 | 10,985-15,676 slots | 16.5-20.9 GiB | 4.5-7.8 GiB | 21.6 | | main + #3729 + points 2-3 | 24,576 slots, every start | none | 6.2-6.5 GiB | 26.3 | - **Refusal (point 3):** with `STRATA_UMA_HEADROOM_GIB=30` the server exited 8 s after it started, right after sizing: "REFUSED: this model does not fit in the memory the integrated GPU shares with the system: 14606 of its experts (about 23.8 GiB) have no room ...". - **Point 4:** 3,197-token prompts at 490 tok/s against 288 with the generic kernels, 12,676 tokens at 510 against 387. Decode is the same or slightly faster. The first token's KL against the generic kernels is below their own run-to-run KL, with the same top-1. One parity test fails: `qsa_prompt_attn_parity`'s int8 cases land at 1.2e-4-1.3e-4 of scale against its 1e-4 (details in the PR). **Tradeoffs.** - With points 2-3 alone, 6K-token prompts are about 15% slower than on main (median 335 against 394 tok/s, 15 and 8 runs). The prompt loan's borrowed experts are read back from the file, where main's mirror holds most of them. Neither the lend mirror nor routed-only staging recovered it. With point 4 as well, the same prompt runs at 582 tok/s. - The headroom stays at 6 GiB. With it, this machine's cache held 23,900-24,576 of the 24,576 experts. - Unlike #2114 (Arc 140T), sizing stays in `device_free_bytes()` where #409 put it, and prompt-slot borrowing stays on. #4712 (Windows, WDDM budget) touches the same function; whichever lands second rebases. **Measured and not proposed:** `STRATA_VERIFY_NO_HOST` by default on an integrated GPU (with 3,115 experts outside the cache, reading them from the file, from a mirror, or from a mirror with `STRATA_VERIFY_NO_HOST=1` all gave correct answers; #515's token-0 reports were on 0.1.54 and 0.1.59), and dropping the start's reads from the file cache (the same speed and memory, a slower fill). **Related:** #3729 (B390 measurements posted there), #2114 and #2115 (@andersson82, Arc 140T: should we merge efforts?), #4712, #6317 and #6319 (Vulkan; this keeps the B390 on SYCL), aef30d8fc (#2110), 928aa3d16 (#6250), #515, #2412, #2282, #409. **Questions.** (a) Are points 1-3 acceptable for integrated GPUs, on top of #3729? (b) Should point 4 be its own PR? **Draft PRs,** each measured against the same build of main: #6453 (points 2-3; on top of #3729, whose commit it carries), #6455 (point 4) and #6454 (the Monitor and the server's free-memory checks on such a GPU). #6454 and #6455 do not depend on #6453 and merge cleanly with it in any order. Two small fixes found on the way, independent of the rest and ready for review: #6451 (`strata launch --model-path` names the wrong shard as `--ple-gguf` for Swift) and #6452 (an on-demand CPU picture reader asks the expert cache for VRAM). All runs and logs: https://github.com/xenonhr/Strata/tree/b390-evidence (README.md lists every file) ### What do you do today instead? A local fork of v0.2.16 with these changes, started from a hand-written run config; `./setup.sh --backend sycl` stops at built-in graphics (`engine/sycl/setup_intel.py`). Without the fork, the B390 runs with the cache plus mirror described above. Prepared with an AI coding assistant (Claude Code); every number above was measured on the machine named above.
Sur le site
Liens install, modèles, releases.