Pull requests / #2495
#2495 sycl: images on the SYCL port (CPU encoder, --vision cpu)
open · @skipzx-lab · 0 comentários · No GitHub
Setup & installServer & APIAMD / HIPNVIDIA / CUDAModels & quantsDocumentation
Descrição
## Title Issue: none. docs/INTEL.md listed images as "Not done" on the SYCL port. ## Summary This turns on images for the Intel SYCL port. The engine already had `--vision` / `GENI` / M-RoPE from the port, and that code works unchanged. This PR adds what was missing around it: a way to build the CPU encoder, setup support, and getting the embeddings file into the engine's container. Images stay opt-in (`--vision cpu`). Without the flag the config is byte-identical to before. ## What changed - `sycl/tools/build-vision.sh` (new): builds `strata-vision` (tools/vision, CPU, the same cmake flags as setup's `build_vision_cpu`) in the `strata-sycl-dev` image. - It copies the image's `libgomp.so.1` next to the binary and writes a launcher, `build-vision/strata-vision.sh`, so a host without an OpenMP runtime can run it. - It prints the image's glibc version. Without `third_party/llama.cpp` it ends with `BUILD EXIT 1` and says what to run. - `sycl/setup_intel.py`: - `--vision cpu` turns images on. `yes` and `gpu` also select the CPU encoder and print a warning, because the port has no GPU encoder. Images stay off by default. - It stops early, with the three steps in order, when images are asked for and the encoder is not built. - `to_sycl()` keeps the `vision` section with host paths (the encoder runs on the host) and the launcher as `exe`, and adds `--vision` exactly once. - Setup's 700 MiB GPU-encoder reserve is dropped unless the user typed it, because the CPU encoder takes no VRAM. - Side fix: the `--vram-reserve-mib` / `--vision` checks in `install()` now see the `--flag=value` form too. Before, `--vision=cpu` or `--vram-reserve-mib=N` got a second copy of the flag added. - `sycl/serve/server_intel.py`: `install_vision_paths()` runs only with a `vision` section. It puts the embeddings file under `/dev/shm/strata-sycl-vision`, or `<data root>/vision-tmp` when `/dev/shm` is short, both mode 0700, and exports `STRATA_SYCL_VISION_DIRS`. - `sycl/serve/strata-sycl.sh`: with `--vision` in the args, it mounts those directories read-only at the same paths. Without `--vision` nothing changes. - `tools/test_setup_sycl_vision.py` (new, 33 tests, no GPU or network) covers: - the mapping and the early stop; - `to_sycl` with and without images, the latter checked against the output of the code before this change; - the reserve rule; - the path redirection, run in temp directories; - the wrapper's mounts, using a fake `docker`; - `bash -n`, and `build-vision.sh` without llama.cpp. - `docs/INTEL.md`: - the feature table entry; - a new "Images (SYCL port)" section covering the build order, how it works, RAM, the image-token cap and measurements; - "Not done" now covers only the llama.cpp path. ## Extra Notes - Measured on an Arc Pro B60 24 GB (xe, PCIe Gen3 x8) with a Xeon W-2155, Coder IQ1_M, `--resident-experts`, and the encoder on 4 threads. - A 1920x1080 dashboard screenshot read correctly in 4.0 s with the cap at 1024 (1,071 tokens). Uncapped it was 2,103 tokens and 5.2 s. - Several images in one message, follow-up turns, PNG/JPEG/WebP, OpenAI and Anthropic image blocks, streaming, and a broken image (400, then the next request reads) all work. - Text-only decode is unchanged. - The IQ1_M model misreads some repeated-digit strings in pictures, for example "2222" read as "22". The pinned llama.cpp's `llama-mtmd-cli` (SYCL, the same GGUF and mmproj) gives exactly the same four wrong answers, so this comes from the model at this quant, not from the port. - Not tested: - an Alchemist card; - a layer split, although the mrope table upload already covers every stage; - setup's default cap of 300 image tokens. I measured at 1024 and uncapped; the docs say so. - Builds: the SYCL engine itself is unchanged, and the encoder was built with `build-vision.sh` in the dev image. No CUDA or HIP files are touched. - Tests: `python -m unittest tools.test_setup_sycl_vision tools.test_setup_sycl tools.test_setup_small_card` gives 67 OK.
No site
Links install, modelos, releases.