#6643 [Engine] strata-inkling: GGUF --native never reaches the CUDA backend ("Invalid .strata magic")
open · @S-Litvinov · 1 Kommentare · Auf GitHub
Server & APIAMD / HIPNVIDIA / CUDAModels & quantsWindows
Beschreibung
[Engine] strata-inkling: GGUF --native never reaches the CUDA backend ("Invalid .strata magic")
## Problem and result
`strata-inkling --serve --native <any .gguf>` on a CUDA build exits before ready:
```
ERR init: Invalid .strata magic (expected STRATA\x00\x01)
```
This is not file-specific: reproduced with `unsloth/Inkling-2-Mini-GGUF/Inkling-2-Mini-UD-Q4_K_XL.gguf`
and with `nokeepai/Inkling-2-Mini-Unbound-v2-Q4_K_M.gguf` on the stock 0.1.65 Windows CUDA
release, and on current main (c6f6427) built with `-DSTRATA_ENABLE_CUDA=ON
-DCMAKE_CUDA_ARCHITECTURES=80`. `--info` on the same files works, and the CPU backend runs
them end-to-end, so the GGUF loader itself is fine - only CUDA never sees it.
## Cause
`InkCudaBackend::init` (engine/src/inkling2/cuda/ink_cuda_backend.cu, the
`model_.load(mf.path, ...)` call) re-opens the `--native` PATH through the .strata artifact
reader. The in-memory `GgufAdapter` assembled by `InkModelFile::open` (manifest, section-1
arena, GGUF_RAW expert bodies via `read_section1`/`read_expert_body`) is ignored, so
`strata_reader.cpp` hits the GGUF magic and fails. The CPU backend consumes the adapter and
works.
## Request
Wire the CUDA expert streaming (the runner already understands `SkCode::GGUF_RAW` bodies in
`inkling2_loop.cpp`) to the adapter when `mf` came from GGUF - or materialize the adapter
output into a temp pack at init. Until then a GGUF run guide for GPU users is impossible.
## Environment
Windows 11, RTX 3090 (sm_80, driver 580+), CUDA 13.3, local build + stock 0.1.65 release
component, both behave identically. Not tried: Metal/HIP.
Mehr auf der Site
Links zu Install, Modellen, Releases.