改动 / #2483
#2483 sycl: images on the Intel port, and the setup wiring for it
open · @beyondcr · 0 评论 · 去 GitHub 看
Setup & installServer & APIMulti-GPUNVIDIA / CUDAModels & quantsDocumentationLinux
说明
## Title
Issue: none; this adds the image encoder to the Intel port (related to #1545, #1606, the Arc A770 setup work
this was measured on, and #2229, which covers the CPU half of the same wiring).
## Summary
On `main`, images are the one feature the CUDA engine has that the Intel port does not. The engine's half of the
path is already ported (`--vision`, the `GENI` request, the per-cell M-RoPE table, the `SVE1` file); what is
missing is the program that turns a picture into those rows.
That program is `strata-vision`, the encoder in `tools/vision`. It is a thin front end over llama.cpp's `mtmd`
and the model's `mmproj` GGUF, with no kernel of Strata's own, and `tools/mtmd/clip.cpp` selects its backend by
device *type*, so llama.cpp's SYCL backend runs the CLIP graph on the Arc with **no source change to the tool**.
What was missing was a build option and a wiring step.
This change adds both, demonstrated end to end on an Arc A770 on `xe`: `tools/vision/CMakeLists.txt` gains
`STRATA_VISION_SYCL`, the encoder gets the launcher a oneAPI binary needs, `sycl/tools/build.sh` builds it
beside the engine, and `sycl/setup_intel.py` stops answering `--vision` with `none`. `docs/INTEL.md`'s
comparison table and its "Not done" section said images were missing on the port; the port's row is now `yes`.
## What changed
Six files, +374/-7, all under `tools/`, `sycl/` and `docs/`. Nothing else is touched.
- `tools/vision/CMakeLists.txt` (+25/-1): `option(STRATA_VISION_SYCL ...)` for the Intel GPU, the matching
`set(GGML_SYCL ... FORCE)` beside the existing `GGML_CUDA` line, and one `if(STRATA_VISION_SYCL)` block that
requires `MKL_DIR` (ggml-sycl's CMake does `find_package(MKL REQUIRED)` for the INTEL target) and prints one
`STATUS` line. The GPU backends are one list (`STRATA_VISION_GPU`), refused when more than one is set. The
option defaults OFF, so a build that does not pass it configures exactly as before.
- `sycl/serve/strata-vision-sycl.sh` (new, 58 lines): the encoder's launcher, in the shape of `strata-sycl.sh`,
the engine's own. A header names its two variables (`STRATA_VISION_BIN`, `STRATA_ONEAPI_ROOT`); the
`LD_LIBRARY_PATH` is composed from the oneAPI install it finds, with no hard-coded paths; an `ldd` preflight
makes a missing runtime fail with a named fix instead of the loader's message; and `exec "$bin" "$@"` passes
every argument the server sends through unchanged.
- `sycl/setup_intel.py` (+83/-3): `sycl_vision_encoder()` (discovers the built encoder the way `sycl_engine()`
discovers the engine), `vision_intel()` (`yes`/`gpu` place the CLIP graph on the Arc, `cpu` runs the same
binary without `--gpu`; anything else, or no encoder, leaves images off with a message), `build_engine_sycl()`
(stages the encoder where the config can name it), and `to_sycl()`, which keeps the config's `vision` section,
names the launcher, and raises `--vram-reserve-mib` to the encoder's own need.
- `sycl/tools/build.sh` (+46): no targets builds the encoder beside the engine, `strata-vision` builds it alone,
`strata` still builds the engine alone. Missing a llama.cpp checkout or oneMKL it says so and the engine still
builds.
- `docs/INTEL.md` (+21/-3): the `images` row of the comparison table, a short "Images on the SYCL port" section
(what the encoder is, what to build, the `--vision` wiring, what the first image costs), and the "Not done"
entry rewritten so it no longer claims images are missing on the port. The section carries no measurement of
this work (no card, no encode times, no comparison against the CPU producer, no end-to-end claim); the numbers
are below, where a reviewer looks for them.
- `tools/test_setup_intel_vision.py` (new, 11 tests): off stays off; `yes`/`gpu` need a built encoder; `cpu` is
the same binary without `--gpu`; the config's `vision` section survives `to_sycl()` and names the launcher;
and the launcher exists, is executable, keeps the encoder's arguments, and finds the oneAPI install from the
`icpx` on `PATH` or from `STRATA_ONEAPI_ROOT`, with the real launcher run against a tree laid out like an
install.
## How it was checked (Arc A770 16 GB, Alchemist, compute capability 12.55, xe, Linux 7.2.6, oneAPI 2026.1 JIT, Ryzen 9 3950X, 64 GB RAM)
Model `Qwen3.8-Flash-Next-Uncensored-IQ3_XXS` with its own `mmproj` (F16, `projection_dim` 2560). Encoder built
against llama.cpp `207bdab`. Every number below was measured on this branch's own binary, on this card.
**The build.** `cmake -S tools/vision -B build-vision-sycl -G Ninja -DCMAKE_BUILD_TYPE=Release
-DLLAMA_DIR=<checkout> -DCMAKE_C_COMPILER=icx -DCMAKE_CXX_COMPILER=icpx -DSTRATA_VISION_SYCL=ON
-DMKL_DIR=<oneMKL cmake dir>` then `--target strata-vision`: **239 targets, 0 errors**. The same configure run
four ways:
| configure | result |
| --- | --- |
| `-DSTRATA_VISION_SYCL=ON -DMKL_DIR=…` | `GGML_SYCL=ON`, `GGML_SYCL_F16=OFF`, `GGML_SYCL_DNN=ON` (oneDNN is not installed, so it is inert), `GGML_CUDA=OFF` |
| `-DSTRATA_VISION_CUDA=ON -DSTRATA_VISION_SYCL=ON` | configure refused: `STRATA_VISION_CUDA;STRATA_VISION_SYCL are exclusive; a build has one GPU backend` |
| `-DSTRATA_VISION_SYCL=ON` with no `MKL_DIR` | configure refused with the message above, before ggml sees it |
| no GPU option | `GGML_SYCL=OFF`, unchanged from today |
**What it encodes.** Against this model's `mmproj`:
```
READY 2560
OK 720 40 18 1063
OK 256 16 16 214
```
720 rows on a 40x18 raster grid, `n_embd=2560`, one `SVE1` record, for a real 1862x1094 screenshot. 256 rows on
16x16 for a 512x512 picture. `nx*ny == n` and the width the engine insists on.
**Two producers, one binary.** The encoder is both producers (with `--gpu` the CLIP graph is on the Arc,
without it on the CPU), which removes the usual confound in a CPU-versus-GPU comparison. Compared record by
record with a local comparator (it reads two `SVE1` files and reports the per-row cosine and the max and mean
absolute difference; it is a measuring instrument, so it is not in the tree):
| picture | grid | header / width | mean abs diff | max abs diff | per-row cosine, mean | min | rows below 0.99 |
| --- | --- | --- | --- | --- | --- | --- | --- |
| 512x512 | 16x16 | identical | 4.028e-04 | 1.522e-02 | **0.999685** | 0.971744 (row 229) | 1 of 256 |
| 1862x1094 (a real screenshot) | 40x18 | identical | 4.010e-04 | 3.088e-02 | **0.999711** | 0.970711 (row 100) | 4 of 720 |
**Numerically close, not bit-exact.** Two controls: the Intel producer is byte-deterministic (two runs,
identical files), and turning flash attention off on the CPU backend alone moves the same rows by 1.0e-02 (max
abs), about two thirds of the 1.5e-02 that the same picture's CPU-and-GPU pair differs by, so the residual is
arithmetic ordering rather than preprocessing. A wrong resize, patch raster order or merge order gives rows that
are unrelated (cosine near zero), not rows that agree to three decimal places.
**Encode time, on the Arc** (this branch's binary, same A770):
| what | time |
| --- | --- |
| process start (device, CLIP graph, projector) | ~1 s |
| the warm-up before `READY` (one 2048x2048 image at the 1024-token cap) | **18.6 s**, once per server start |
| a real 1862x1094 screenshot (720 rows, 40x18) | **1.06 s** (the CPU backend: 5.02 s) |
| a 512x512 picture (256 rows, 16x16) | **0.21 s** (the CPU backend: 1.39 s) |
The warm-up is almost all one-time JIT compilation of the CLIP graph, not the picture. No performance claim is
made: the encoder is not on the critical path (one encode per picture, cached by the image's hash for later
turns), and nothing about the text model's speed changes.
**End to end.** `--vision` reaches the engine, the encoder starts before it (so its device buffers are allocated
before the engine sizes its expert cache), and a fresh OpenAI image request about the real screenshot returns
the right two facts about it:
```
$ python3 ask_image.py 8096 wiki-intel-arc.png "Transcribe the page title at the top of this screenshot, and
name the company the sidebar box says designed the product."
Page title: **Intel Arc**
Designed by (per the sidebar infobox): **Intel**
usage: prompt 798 tokens (78 text + the encoder's 720 image rows), completion 139
```
The request ran against a server whose `vision.exe` is this branch's launcher and whose encoder is this branch's
own build.
**The launcher's preflight, four ways.** With no oneAPI install reachable (no `LD_LIBRARY_PATH`, no `icpx` on
`PATH`, no `/opt/intel/oneapi`) it exits 1 with the one fix to apply. The capture is
`evidence/fresh/launcher-preflight-four-ways-runtime-root.txt`, run on this commit; the three
`launcher-preflight-both-ways*` files beside it are the earlier revisions, kept as they were taken and labelled
(the revision before the last review pass, the three-commit tip, the run before the launcher was renamed):
```
strata-vision: the image encoder cannot start: missing runtime libraries: libsycl.so.9
strata-vision: install Intel oneAPI (the DPC++ compiler and oneMKL components), then either
strata-vision: source <oneapi>/setvars.sh # before starting the server, or
strata-vision: export STRATA_ONEAPI_ROOT=<oneapi-install-root>
```
It preflights clean and execs the encoder in the other three cases: with `STRATA_ONEAPI_ROOT` set to the oneAPI
tree; with nothing set but an install at Intel's own default `/opt/intel/oneapi`; and with the `icpx` on `PATH`
and the loader path otherwise empty, the route that takes the root from the compiler's own location. An unknown
argument reaches the encoder's own parser (`unknown argument --bogus`, exit 2, device untouched), which proves
resolution without using the card. On the card the encoder runs to `READY 2560` and `OK` records.
## Extra Notes
- **Scope.** An Arc A770 on `xe` only: not Battlemage or a B-series, not an A750 on `i915`, not a two-card box.
`ONEAPI_DEVICE_SELECTOR` is the device switch, moot on one Intel card.
- **The installer on a host with a system oneAPI install.** The build behind this change takes the encoder's
toolchain from the port's container overlay; at run time the launcher finds oneAPI from `icpx` on `PATH`, from
`STRATA_ONEAPI_ROOT`, or from Intel's own default `/opt/intel/oneapi`.
- **Not bit-exact** (see the table). A row can differ in the last places. Use is unaffected: the engine reads the
rows as the picture and answers about it correctly, and a token that diverges against llama.cpp is not excluded.
- **Why the encoder has a launcher of its own.** `sycl/serve/strata-sycl.sh` is the engine's launcher and runs
the engine inside the port's dev image, so it cannot start the encoder. `serve/server.py` starts the config's
`vision.exe` itself, as a host process, where the container's oneAPI runtime is not on the loader path;
`strata-vision-sycl.sh` is the launcher that supplies it, named the way `strata-sycl.sh` is, the binary it
starts then the backend that binary was built for.
- **#2229 touches the same file.** That pull request wires the CPU encoder and declines `yes`/`gpu` with a
warning; this one adds the GPU path and the setup wiring around it, so both change `sycl/setup_intel.py`.
Whichever lands first, the other needs a small rebase. #2229 also remaps the embeddings path for the container
deployment (`sycl/serve/server_intel.py`, `install_embeddings_path`), which this change does not close.
- **The one maintenance cost.** A second llama.cpp build now sits beside the port, a ggml + `mtmd` stack at the
pinned commit, which is why the encoder cannot be built by the port's own CMake. The encoder is built against
llama.cpp `207bdab` here while the port pins `3cf03257` (the engine's `CMakeLists.txt` `GIT_TAG`, the revision
recorded in `third_party/ggml/VERSION.txt`), so a llama.cpp bump has to move both builds and re-run the
comparison. Everything else added is opt-in: `--vision` is off unless setup is asked for images, and no
existing path changes shape if the 本站相关内容
相关页面的快捷入口。