改动 / #2817

#2817 sycl: setup --vision cpu reads images on the Intel port (the CPU encoder, as on AMD)

open · @hotschmoe · 0 评论 · 去 GitHub 看

BenchmarksSetup & installServer & APIMulti-GPUAMD / HIPNVIDIA / CUDAModels & quantsDocumentation

说明

## Title
sycl: setup --vision cpu reads images on the Intel port (the CPU encoder, as on AMD)

Issue: Resolves #2816

## Summary

Images on the Intel (SYCL) port, the way the AMD (HIP) path has them since #304: the image encoder built for the
CPU by setup, the engine started with `--vision`. docs/INTEL.md's comparison table said "not yet" for images on
the SYCL port, but the engine side has been there since the port (`--vision`, the GENI request with the encoder's
embeddings, the per-cell M-RoPE position table in `sycl/src/program/generate.cpp`); what was missing was the setup
wiring. `sycl/setup_intel.py` forced `--vision none`, mapped every `--vision` to "none" and dropped the config's
`"vision"` section. The encoder itself (`tools/vision`, llama.cpp's mtmd + the mmproj file) needs no SYCL: the
CPU build the AMD path uses runs on the PC, outside the container, and `serve/server.py` hands the engine the
embeddings.

With this change `./setup.sh --backend sycl --vision cpu` compiles the encoder for the CPU into `engine-sycl/`
(setup's own `build_vision_cpu`: `tools/vision` against the pinned llama.cpp, `-DSTRATA_VISION_CUDA=OFF
-DSTRATA_PORTABLE=OFF`, once, again when `tools/vision` changes), downloads the mmproj file as setup does, keeps
the config's `"vision"` section (`exe`, `mmproj`, `model`, `threads`: host paths, since the encoder runs on the
host) and `--vision`, and applies the port's own VRAM reserve rule (1,024 MiB up to 32K, ...) instead of the
700 MiB setup writes for the CUDA engine's GPU encoder: the CPU encoder takes no VRAM. A `--vram-reserve-mib`
the user gives is kept. `--vision yes` / `gpu` leave images off with a note saying to use `--vision cpu`, as
`hip_vision` does on AMD (there is no SYCL encoder build). Without `--vision cpu` nothing changes: the
throw-away engine folder and the config are as before.

Measured, 2x Intel Arc Pro B70 32 GB (kernel 7.1, Compute Runtime 26.22.38646.4, oneAPI 2026.1, AOT bmg-g31),
AMD Threadripper 1950X, Qwen3.8-Flash-Next ISTA GSQ-RCO IQ3_XXS, `--layer-split 30 --kv int8 --spec 4 --mtp
<BF16 draft layer> --mtp-max-t 4`, the encoder from `tools/vision` built with gcc 13 (native,
`-DSTRATA_PORTABLE=OFF`), `mmproj-Qwen3.8-Flash-Next-BF16.gguf`, 8 threads:

- the encoder alone: a 256x256 PNG -> 64 image tokens (an 8x8 grid, 2,560 values each), 1,191 ms on 8 CPU
  threads, no VRAM.
- a 448x448 image of four coloured quadrants, "name the colour of each quadrant", through
  `/v1/chat/completions` on a development tree with the same `serve/` and vision path as `main`: all four
  named correctly ("Top-left: red, top-right: green, bottom-left: blue, bottom-right: yellow"; 280 prompt
  tokens, 4.8 s to the first token including the encode), also while a code request ran on the other slot; a
  512-token text reply decoded at 140.9 and 140.5 tok/s with `--vision` (about the 141 measured without it) on that tree.
- on `main` 296b59d2 (upstream's own engine build, this branch's config: the vision section and `--vision`):
  the four colours right ("Red, Green, Blue, Yellow" to "in the order top-left, top-right, bottom-left,
  bottom-right"), 246 prompt tokens with the picture read in 1.48 s including the encode; a 512-token greedy
  text reply at 81.7 / 79.4 tok/s with `--vision` against 81.8 / 79.4 without (an earlier run of the same binary
  and config), with identical text and draft counts: `--vision` changes nothing for text.
- `python -m unittest tools.test_setup_sycl`: 24 tests pass (6 new). Every `tools/test_setup_*.py` module run
  with and without this change gives the same result (the same 10 pre-existing failures on `main`: the
  `test_setup_golden` UD-Q4_K_XL cases and two `test_setup_engine_hash` cases; none in the SYCL, AMD, config or
  bench-tip modules).

## What changed

- `sycl/setup_intel.py`:
  - `sycl_vision(asked)` replaces `S.hip_vision` (was `lambda asked: "none"`): `cpu` -> `cpu`; `yes` / `gpu` ->
    `none` with a warning that says `--vision cpu`; anything else `none`.
  - `build_engine_sycl(gpu, llama, vision)` replaces `S.build_engine_hip` (was the stub): with `vision == "none"`
    it returns the same throw-away stub as before (`engine_stub()`, the code moved out of `install()`); with
    `cpu` it stops with the package to install when the PC has no C++ compiler, and otherwise calls setup's
    `build_vision_cpu(eng, stamp, meta, llama, vsrc)` with `eng = engine-sycl/` (new folder next to the
    checkout's `engine/`, which the SYCL setup never used): that compiles `tools/vision` into `build-vision/`,
    copies `strata-vision` beside the stamp and records `vision: cpu` and the `tools/vision` source hash in
    `engine-sycl/BUILD.json`, so a rerun compiles nothing. `build_vision_cpu` is added to the names
    `install()` checks setup.py still has.
  - `to_sycl`: the `"vision"` key is no longer dropped from the config; `--vision` was already passed through.
    The reserve: when `--vision` is in the args and the reserve is setup's vision default (`VISION["cpu"]
    ["reserve_mib"]`, 700) it is treated as not given, which is what setup's own `choices_from_config` does
    with that value (#493), so the port's rule applies.
  - `say_intel` rewrites `build_vision_cpu`'s "engine: engine-sycl/strata, image encoder (CPU): ..." line to
    "image encoder (CPU): ..." (there is no engine in that folder).
  - `install()` still appends `--vision none` when `--vision` is not given: images stay opt-in.
- `tools/test_setup_sycl.py`: `ToSycl.test_the_vision_section_is_kept_and_the_cpu_encoder_takes_no_reserve`
  and a new `Images` class (the vision choice; no images -> the stub and no `cmake_build` call; `cpu` -> one
  `cmake_build` of `tools/vision` with the AMD path's definitions, the copied encoder, the stamp, no second
  build on a rerun, a build again when `tools/vision` changes; no compiler -> a stop naming `build-essential`;
  the source still forces `--vision none` when not asked). No GPU, no network, no compiler: `cmake_build` is
  replaced.
- `docs/INTEL.md`: the comparison table's images row, an "Images" bullet in the setup section, the "Images: the
  encoder on the CPU" section with the measurements above, and the "Not done" entry narrowed to the llama.cpp
  configuration.

Not changed: `setup.py` (`git diff main -- setup.py` is empty; the CUDA and HIP paths run the same code), the
engine (`sycl/src`, `src`, `include`: no compiled file is touched, the binary is `main`'s), `serve/`,
`sycl/serve/strata-sycl.sh`, `tools/vision`. A SYCL build of the encoder (GPU encode) is not part of this: the CPU
encoder is what runs on AMD too.

## Extra Notes

- The encoder runs on the host, not in the `strata-sycl-dev` container, so the PC needs a C++ compiler
  (`build-essential`); setup's pip-installed cmake and ninja do the rest. A PC without one gets a message with
  the package and the `--vision no` way out. The measured encoder above was built with the dev image's gcc 13
  and run inside the same image (our serving setup keeps the server in the container); the setup path builds it
  with the host's compiler - the same CMake definitions.
- Prepared with AI assistance (Claude Code): the code, the measurements and this text were produced by Claude on the submitter's two Arc Pro B70s, at the submitter's request.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

本站相关内容

相关页面的快捷入口。