Issues / #2816
#2816 [Feature]: images on the Intel (SYCL) port through setup, with the CPU image encoder as on AMD
open · @hotschmoe · 0 comentarios · En GitHub
BenchmarksSetup & installServer & APIMulti-GPUAMD / HIPModels & quantsDocumentation
Descripción
### What do you want to do?
Send pictures to Strata on an Intel Arc, through the setup the port already has: `./setup.sh --backend sycl
--vision cpu`, the way `--vision cpu` works on AMD since #304.
docs/INTEL.md's comparison table says images are "not yet" on the SYCL port, and `sycl/setup_intel.py` makes it
so: it forces `--vision none` (`install()`), maps every `--vision` choice to "none" (`S.hip_vision = lambda asked:
"none"`) and drops the config's `"vision"` section in `to_sycl()`. But the engine side is there: the SYCL engine
takes `--vision` (the GENI request with the encoder's embeddings, the per-cell M-RoPE position table in
`sycl/src/program/generate.cpp`), and the image encoder (`strata-vision`, `tools/vision`: llama.cpp's mtmd and the
model's mmproj file) is a CPU program with no SYCL in it - the very build the AMD path makes for the CPU, run by
`serve/server.py` on the host.
The workaround I use today: build `tools/vision` for the CPU by hand (`cmake -S tools/vision -B build-vision -G
Ninja -DCMAKE_BUILD_TYPE=Release -DLLAMA_DIR=<pinned llama.cpp> -DSTRATA_PORTABLE=OFF`), then add the `"vision"`
section (`exe`, `mmproj`: `mmproj-Qwen3.8-Flash-Next-BF16.gguf`, `model`: shard 1, `threads`) and `--vision` to the
config by hand. With that, on two Arc Pro B70s (Qwen3.8-Flash-Next ISTA IQ3_XXS), a 448x448 picture of four
coloured quadrants is answered correctly through `/v1/chat/completions`: on `main` 296b59d2 with `--layer-split 30`
("Red, Green, Blue, Yellow" in the asked order), and on a development tree (expert parallel) also next to a
concurrent code request; the encoder takes 1.2 s for a 256x256 test picture on 8 CPU threads (64 image tokens) and no
VRAM; text decode speed is unchanged with `--vision`
(140.9 vs 140.5 tok/s on a 512-token reply, a development tree with the same vision path as `main`).
What I would like: setup does the above for `--backend sycl --vision cpu` (compile the encoder for the CPU, get
the mmproj file, write the vision section and `--vision`), the docs say images work on the port and what was
measured, and nothing changes for a setup run without `--vision cpu`. `--vision yes` (the GPU encoder) can stay off
with a note, as on AMD. A pull request that refers to this issue does that in `sycl/setup_intel.py`,
`tools/test_setup_sycl.py` and `docs/INTEL.md`; `setup.py` itself is not touched.
Prepared with AI assistance (Claude Code): the analysis, the measurements and this text were produced by Claude on the submitter's two Arc Pro B70s, at the submitter's request.
En el sitio
Enlaces a install, modelos, releases.