Issues / #2816
#2816 [Feature]: images on the Intel (SYCL) port through setup, with the CPU image encoder as on AMD
open · @hotschmoe · 0 Kommentare · Auf GitHub
BenchmarksSetup & installServer & APIMulti-GPUAMD / HIPModels & quantsDocumentation
Beschreibung
### What do you want to do?
Send pictures to Strata on an Intel Arc, through the setup the port already has: `./setup.sh --backend sycl
--vision cpu`, the way `--vision cpu` works on AMD since #304.
docs/INTEL.md's comparison table says images are "not yet" on the SYCL port, and `sycl/setup_intel.py` makes it
so: it forces `--vision none` (`install()`), maps every `--vision` choice to "none" (`S.hip_vision = lambda asked:
"none"`) and drops the config's `"vision"` section in `to_sycl()`. But the engine side is there: the SYCL engine
takes `--vision` (the GENI request with the encoder's embeddings, the per-cell M-RoPE position table in
`sycl/src/program/generate.cpp`), and the image encoder (`strata-vision`, `tools/vision`: llama.cpp's mtmd and the
model's mmproj file) is a CPU program with no SYCL in it - the very build the AMD path makes for the CPU, run by
`serve/server.py` on the host.
The workaround I use today: build `tools/vision` for the CPU by hand (`cmake -S tools/vision -B build-vision -G
Ninja -DCMAKE_BUILD_TYPE=Release -DLLAMA_DIR=<pinned llama.cpp> -DSTRATA_PORTABLE=OFF`), then add the `"vision"`
section (`exe`, `mmproj`: `mmproj-Qwen3.8-Flash-Next-BF16.gguf`, `model`: shard 1, `threads`) and `--vision` to the
config by hand. With that, on two Arc Pro B70s (Qwen3.8-Flash-Next ISTA IQ3_XXS), a 448x448 picture of four
coloured quadrants is answered correctly through `/v1/chat/completions`: on `main` 296b59d2 with `--layer-split 30`
("Red, Green, Blue, Yellow" in the asked order), and on a development tree (expert parallel) also next to a
concurrent code request; the encoder takes 1.2 s for a 256x256 test picture on 8 CPU threads (64 image tokens) and no
VRAM; text decode speed is unchanged with `--vision`
(140.9 vs 140.5 tok/s on a 512-token reply, a development tree with the same vision path as `main`).
What I would like: setup does the above for `--backend sycl --vision cpu` (compile the encoder for the CPU, get
the mmproj file, write the vision section and `--vision`), the docs say images work on the port and what was
measured, and nothing changes for a setup run without `--vision cpu`. `--vision yes` (the GPU encoder) can stay off
with a note, as on AMD. A pull request that refers to this issue does that in `sycl/setup_intel.py`,
`tools/test_setup_sycl.py` and `docs/INTEL.md`; `setup.py` itself is not touched.
Prepared with AI assistance (Claude Code): the analysis, the measurements and this text were produced by Claude on the submitter's two Arc Pro B70s, at the submitter's request.
Mehr auf der Site
Links zu Install, Modellen, Releases.