Pull requests / #1768

#1768 Windows AMD: image input on the CPU and on the GPU through Vulkan, te…

open · @WeisTekEng · 0 comentarios · En GitHub

BenchmarksSetup & installServer & APIAMD / HIPNVIDIA / CUDAModels & quantsDocumentationWindowsLinux

Descripción

## Title
Windows AMD: image input on the CPU and on the GPU through Vulkan (tested on RX 9070 XT Nitro+)

Issue: N/A (AMD on Windows had no way to enable images)

## Summary
AMD cards on Windows could not use images: setup refused `--vision cpu`, never asked the "Images?" question, and the ready-made HIP engine ships no `strata-vision.exe`. The encoder was also CUDA-only, so there was no GPU option on AMD. This PR lets AMD users on Windows (and Linux) choose no images, the CPU encoder, or a GPU encoder built with Vulkan, compiled locally once.

Tested on an RX 9070 XT Nitro+ (gfx1201), Windows, Ryzen 9 5900X, 80 GB RAM, Swift 1.5 IQ3_XXS, Visual Studio 2026 and the Vulkan SDK. Setup with `--vision gpu` compiled the encoder and the server started with images on. The model correctly read `TS20 CP77` and `QUADRA` from a test picture. Text speed stayed at about 50 tok/s with a 70-76% expert-cache hit rate, with and without images. Unit tests pass (102 in `test_setup_amd`, `test_setup_choices`, `test_strata_mcp`).

## What changed
- `setup.py`
  - `hip_vision()` now accepts `cpu` on Windows and returns `gpu` for `gpu`/`yes` (Vulkan encoder) instead of refusing.
  - New `win_hip_vision()`: compiles the encoder once beside the ready-made AMD engine. If the Visual Studio C++ tools or the Vulkan SDK are missing it warns and continues with images off.
  - New `build_vision_vulkan()` and `find_vulkan_sdk()`.
  - Windows AMD setup asks "Images?" (no / CPU / GPU through Vulkan).
  - `build_engine_hip()` dispatches to the CPU or Vulkan encoder build, and its readiness check compares the encoder mode so switching cpu to gpu rebuilds it.
  - The encoder builds call `find_vcvars((13, 3))`, because the Visual Studio version cap exists for CUDA and doesn't apply here. This lets Visual Studio 2026 work.
- `tools/vision/CMakeLists.txt`: new `STRATA_VISION_VULKAN` option (builds ggml-vulkan).
- `tools/strata_mcp.py`: removed the check that rejected `--vision` choices for the AMD backend.
- `tools/test_setup_amd.py`, `tools/test_setup_choices.py`, `tools/test_strata_mcp.py`: updated and added tests.
- `docs/AMD_HIP.md`: images are now supported on Windows AMD.
- `.gitignore`: added the generated `build-vision-vk.bat`.

## Extra Notes
- Compiling needs the Visual Studio C++ tools, and for GPU vision the Vulkan SDK. The first build takes 10-20 minutes. `--vision cpu` needs no SDK.
- Only tested on one machine. Not tested: Linux AMD with the Vulkan encoder, other Radeon cards, or a PC with an integrated GPU. On such a PC Vulkan may pick the wrong device, and `GGML_VK_VISIBLE_DEVICES` may help, but I haven't verified that.
- The install tool in `strata_mcp.py` still rejects AMD on Windows. I left that out of this PR.

En el sitio

Enlaces a install, modelos, releases.