#6500 [Engine] Vulkan backend: scope for the port (stage 1 landed: the loop and one Q4_0 kernel)

closed · @jomatech-vdc · 1 commentaires · Sur GitHub

Setup & installAMD / HIPNVIDIA / CUDAModels & quantsLinux

Description

### The problem
The engine has no GPU backend that builds for an Intel Mac, and none for any card outside CUDA/HIP/SYCL/Metal. A Vulkan backend covers both. On Macs the driver half is settled — MoltenVK for Intel Macs (Khronos still recommends it there), KosmicKrisp on Apple Silicon (Khronos-conformant Vulkan 1.4, needs macOS 26+) — the missing half is engine code: our ggml is a base+CPU fork and every GPU path is our own kernels, so `-DGGML_VULKAN=ON` has nothing to flip. It is a port, and this issue agrees its scope.

### Result so far (stage 1, this repo)
`engine/vulkan/` — separate CMake project like metal/ and sycl/, default path untouched: GLSL → SPIR-V at build time (glslangValidator, `--target-env vulkan1.1`), instance with `VK_KHR_portability_enumeration` (what makes MoltenVK/KosmicKrisp devices enumerable), one kernel `q40_dot.comp` (Q4_0 block · float32 row) compared block-for-block against the CPU oracle, device name printed. Built and ctest-registered here.

Measured/not measured: the dev box's Vulkan drivers are broken (Mesa 26.3.1 ANV segfaults inside pipeline creation for any shader, even an empty one; the NVIDIA ICD there fails instance creation). So stage 1's numbers are NOT yet measured on any device — first real run belongs on a healthy NVIDIA/AMD box (the RTX 4090 rig). Marking this per CONTRIBUTING: driver faults observed on our hardware; kernel parity itself is unconfirmed until a device runs it.

### Proposed stages
1. done: the loop + one Q4_0 dot kernel with oracle test.
2. decode-side parity set: quant dots for the served packs (Q4_K/Q8_0/IQ*), dequant, fused dot+scale, KV paths — each with a CPU parity test, like the Metal port's order.
3. prefill side (our prefills are CUDA-only today).
4. expert streaming over Vulkan memory models; then `--backend vulkan` in the server and the choice in `choose_mac_backend` / setup for cards a native backend misses.
5. measure through MoltenVK (Intel Mac) and KosmicKrisp (Apple Silicon, macOS 26+); only then flip the Mac choice from the stated refusal to vulkan where it wins.

Sizes for expectations: engine/metal ~27k lines, engine/sycl ~165k. This is weeks-to-months of work; every stage keeps the default path byte-identical.

### Validation so far
- passed: build + ctest registration (stage 1) on CachyOS/Linux; all other suites unaffected (no root-graph change; root CMakeLists touched in comments only).
- failed: on-device run — the dev box's ANV driver segfaults on any pipeline creation (driver regression, noted, not our code).
- not run: everything stage 2 and later.

Sur le site

Liens install, modèles, releases.