Pull requests / #2240
#2240 Remote stage: a worker for the SYCL build (Intel Arc), stacked on #1748; STRATA_LOGPOS_IDS for cross-engine KL
open · draft · @YukihiroHemmi · 0 Kommentare · Auf GitHub
BenchmarksServer & APIAMD / HIPNVIDIA / CUDAModels & quantsDocumentationWindows
Beschreibung
Stacked on #1748 (lask3802's remote stage; the first 13 commits are that PR's head 4bbcb16e, unchanged). This adds a **worker for the SYCL build** (Intel Arc) and two measurement-only environment variables. Nothing changes unless `--stage-worker` is given to the SYCL build, or `STRATA_LOGPOS_IDS` / `STRATA_LOGPOS_SPLIT_AT` are set. ## What it changes - **`sycl/src/program/generate.cpp`**: the SYCL engine can run as a stage worker (`--serve --stage-worker PORT --stage-begin K --stream-experts`): layers [K, 48) with all their experts in VRAM (read from the GGUF; it stops at start with the count if they do not fit), its own dense weights and session, no head, MTP, PLE table or embedding table. The main process is the CUDA or HIP build of #1748. A relay worker, `--remote-stage` and `--vision` are refused with a message on the SYCL build. The SYCL `Verifier` gets `set_no_head` (the hand-off after the last layer), `GgufExpertSource` gets `set_layer_range`; `src/net/stage_link.cpp` is used as it is (`sycl/CMakeLists.txt`). - **`include/strata/core/logpos_ids.hpp`** (new, shared): `STRATA_LOGPOS_IDS=<baseline dump>` makes `STRATA_LOGPOS` also write the log-probabilities of the baseline's top-k ids at each position, so two engines' distributions can be compared by KL without the ids drifting out of the top-k; `STRATA_LOGPOS_SPLIT_AT=N` reads [0, N) of a prompt as chunks and the rest as windows (both trees). Off by default; `src/core/logpos_ids_test.cpp` covers the header. - **Docs**: `docs/REMOTE_STAGE.md` (an "Intel Arc as a worker" section with the numbers below), `docs/INTEL.md` (a note). ## What it leaves alone Every CUDA/HIP source of #1748 except the one-line `logpos::write_ids` call in `src/core/verify.cpp` and the `STRATA_LOGPOS_SPLIT_AT` branch in `src/program/generate.cpp`; both are inert without the variable. No defaults change. ## How the default path was checked On trainsv (Ryzen 9 5900X, RTX 4070, Arc Pro B70, Ubuntu 24.04), builds of 4bbcb16e and of this branch, started without any of the new options, wrote byte-identical `STRATA_LOGPOS` dumps (5 texts, 1,618 positions, `--short-read 1000000 --adapt-every 100000`) for both the CUDA engine (4070) and the SYCL engine (B70). `stage_link_test` passes on both builds; `logpos_ids_test` passes. ## Measured (2026-10-13, engine 0.1.43.1) - **One PC, loopback** (4070 main process, B70 worker, Qwen3.8-Flash-Next IQ2_XS, K=12, the worker's 18,432 experts all in VRAM): decode 73-109 tok/s vs 44-74 on the B70 alone (which mirrors 7,296 experts in pinned RAM) and 26-32 on the 4070 alone with `--mmap-experts`; prompt 4K 1,291 vs 1,204 tok/s. Teacher-forced top-20 logprobs over 12 raw texts (25,256 positions): mean KL 0.135 against the B70 alone, with the "B70 alone vs 4070 alone" floor at 0.152. - **Two PCs over 2.5 GbE** (RTX 5070 Ti main process on another PC, the B70 worker, OrcaRouter IQ3_XXS, K=23): decode 78-108 tok/s vs 81-104 on the 5070 Ti alone (same flags); prompt 4K 1,145 vs 2,244 (a 4,096-token chunk crosses in 2.9 s); a window's round trip 17.2 ms = worker 16.1 + link 1.1; KL against the 5070 Ti alone 0.015 (5 texts, 1,363 positions). - Not run: HIP as the main process, a relay through a SYCL worker, images. The two dumps compared with `STRATA_LOGPOS_IDS` must read the same texts in the same order with the same `STRATA_LOGPOS_SPLIT_AT`: text boundaries are inferred from the position going back to 0, so a text that writes no rows is skipped on both sides alike. Raw data and the measurement scripts (`kl_run.py`, `kl_check.py`, `speed_bench.py`) are available on request. YukihiroHemmi@🐜
Mehr auf der Site
Links zu Install, Modellen, Releases.