#6500 [Engine] Vulkan backend: scope for the port (stage 1 landed: the loop and one Q4_0 kernel)
closed · @jomatech-vdc · 1 Kommentare · Auf GitHub
Setup & installAMD / HIPNVIDIA / CUDAModels & quantsLinux
Beschreibung
### The problem The engine has no GPU backend that builds for an Intel Mac, and none for any card outside CUDA/HIP/SYCL/Metal. A Vulkan backend covers both. On Macs the driver half is settled — MoltenVK for Intel Macs (Khronos still recommends it there), KosmicKrisp on Apple Silicon (Khronos-conformant Vulkan 1.4, needs macOS 26+) — the missing half is engine code: our ggml is a base+CPU fork and every GPU path is our own kernels, so `-DGGML_VULKAN=ON` has nothing to flip. It is a port, and this issue agrees its scope. ### Result so far (stage 1, this repo) `engine/vulkan/` — separate CMake project like metal/ and sycl/, default path untouched: GLSL → SPIR-V at build time (glslangValidator, `--target-env vulkan1.1`), instance with `VK_KHR_portability_enumeration` (what makes MoltenVK/KosmicKrisp devices enumerable), one kernel `q40_dot.comp` (Q4_0 block · float32 row) compared block-for-block against the CPU oracle, device name printed. Built and ctest-registered here. Measured/not measured: the dev box's Vulkan drivers are broken (Mesa 26.3.1 ANV segfaults inside pipeline creation for any shader, even an empty one; the NVIDIA ICD there fails instance creation). So stage 1's numbers are NOT yet measured on any device — first real run belongs on a healthy NVIDIA/AMD box (the RTX 4090 rig). Marking this per CONTRIBUTING: driver faults observed on our hardware; kernel parity itself is unconfirmed until a device runs it. ### Proposed stages 1. done: the loop + one Q4_0 dot kernel with oracle test. 2. decode-side parity set: quant dots for the served packs (Q4_K/Q8_0/IQ*), dequant, fused dot+scale, KV paths — each with a CPU parity test, like the Metal port's order. 3. prefill side (our prefills are CUDA-only today). 4. expert streaming over Vulkan memory models; then `--backend vulkan` in the server and the choice in `choose_mac_backend` / setup for cards a native backend misses. 5. measure through MoltenVK (Intel Mac) and KosmicKrisp (Apple Silicon, macOS 26+); only then flip the Mac choice from the stated refusal to vulkan where it wins. Sizes for expectations: engine/metal ~27k lines, engine/sycl ~165k. This is weeks-to-months of work; every stage keeps the default path byte-identical. ### Validation so far - passed: build + ctest registration (stage 1) on CachyOS/Linux; all other suites unaffected (no root-graph change; root CMakeLists touched in comments only). - failed: on-device run — the dev box's ANV driver segfaults on any pipeline creation (driver regression, noted, not our code). - not run: everything stage 2 and later.
Mehr auf der Site
Links zu Install, Modellen, Releases.