#6645 [Engine] strata-inkling: --kv k8v8 and --kv k8v4 SIGSEGV immediately after load on Linux RTX 3090 (sm_86)

open · @danbunnell · 0 comentarios · En GitHub

BenchmarksNVIDIA / CUDAModels & quantsLinux

Descripción

### Area

[Engine]  (speed, kernels, models, the server and API)

### Desktop app, Terminal or source?

Other

### What happened

Running `strata-inkling --bench` with `--kv k8v8` or `--kv k8v4` on a Linux RTX 3090 (`sm_86`) switches to the `[inkling2-ep]` backend and immediately exits with `SIGSEGV` (`rc=-11`) right after printing the `Loaded` line.

This reproduces on both:
- Official `Inkling-2-Mini-SQ-15.strata` (all `3072 VRAM / 0 host` experts loaded cleanly on GPU0, ~14.6 GiB total VRAM used) across `--prompt-len 128` through `245760`, both `--draft off` and `--draft dspark`, and both `--kv k8v8` and `--kv k8v4`.
- `unsloth/Inkling-2-Mini-GGUF/Inkling-2-Mini-UD-IQ4_XS.gguf` (`2688 VRAM / 384 host` experts, with or without `--vram-reserve-mib 512` / `--cpu-threads 16`).

By contrast, `--selfcheck --native Inkling-2-Mini-SQ-15.strata --kv k8v8` passes (`PASS backend=cuda-ep1`), and default `--kv fp16` / `--kv fp8` (which use `[InkCUDA]`) run `--bench` cleanly at short/32K contexts.

Expected: `--kv k8v8` and `--kv k8v4` run `--bench` without crashing on `sm_86` (this is also needed to fit 128K–256K+ contexts on 24 GB cards alongside #6609).

### Smallest steps or command that reproduce it

1. Download `strata-inkling-0.1.65-linux-x86_64-cuda-cuda13.tar.gz` from `v0.2.19` and `Inkling-2-Mini-SQ-15.strata`.
2. Run:
```sh
LD_LIBRARY_PATH=./bin ./bin/strata-inkling --bench --native ./Inkling-2-Mini-SQ-15.strata --kv k8v8 --draft off --prompt-len 128 --gen 128 --parallel 1
```
3. Same crash with `--kv k8v4`.

### Reproduction status

Reproduced locally

### Strata version

Strata 0.2.19 · strata-inkling 0.1.65 (commit c6f6427d9de9, linux-x86_64-cuda-cuda13)

### Graphics card, memory and system

RTX 3090 24 GB (sm_86) · 32 GB RAM · Linux x86_64

### Model, size and settings

Inkling-2-Mini SQ-15 (14.35 GB) and UD-IQ4_XS GGUF · --kv k8v8 and --kv k8v4 · --parallel 1 · --draft off / dspark

### Logs

```text
[inkling2-ep] Loaded strata (SQ-15) | gpus=1 (p2p=pinned, ep_split=0.50) | experts: 3072 VRAM (GPU0=11.44 GiB, GPU1=0.00 GiB), 0 host (doorbell=off, numa=1, thr=0) | kv=k8v8
# exits with signal 11 (SIGSEGV), peak VRAM 14,977 MiB
```

Same with `--kv k8v4`:
```text
[inkling2-ep] Loaded strata (SQ-15) | gpus=1 (p2p=pinned, ep_split=0.50) | experts: 3072 VRAM (GPU0=11.44 GiB, GPU1=0.00 GiB), 0 host (doorbell=off, numa=1, thr=0) | kv=k8v4
# exits with signal 11 (SIGSEGV), peak VRAM 14,976 MiB
```

With `UD-IQ4_XS.gguf`:
```text
[inkling2-ep] Loaded gguf (GGUF-NVFP4) | gpus=1 (p2p=pinned, ep_split=0.50) | experts: 2688 VRAM (GPU0=11.19 GiB, GPU1=0.00 GiB), 384 host (doorbell=off, numa=1, thr=0) | kv=k8v8
# exits with signal 11 (SIGSEGV), peak VRAM 17,534 MiB
```

En el sitio

Enlaces a install, modelos, releases.