#6569 [Engine] Inkling-2-Mini SQ-10 segfaults in CPU expert computation on RTX 4070 SUPER

open · @mskutin · 2 コメント · GitHub で見る

BenchmarksSetup & installServer & APIAMD / HIPNVIDIA / CUDAModels & quantsLinux

本文

### Area

[Engine]

### Desktop app, Terminal or source?

Terminal (the `strata` command)

### What happened

Inkling-2-Mini SQ-10 loaded and completed the one-token startup check. On a 7,424-token request within the configured 8,192-token context, the engine exited with `-11` (`SIGSEGV`) before generating a reply. The expected result is that the engine processes the prompt and returns a reply.

A standalone benchmark reproduced the segfault with a synthetic 128-token prompt, FP16 KV, and drafting disabled. It exited with status `139` before producing a result.

### Smallest steps or command that reproduce it

With SQ-10 already installed, run the packaged engine directly:

```bash
strata_dir="$HOME/.local/share/strata"
engine="$strata_dir/components/shapes/inkling2/engine/0.1.65-linux-x86_64-cuda-cuda13/strata-inkling"
model="$strata_dir/Strata-data/models/inkling-2-mini/Inkling-2-Mini-SQ-10.strata"

(
  ulimit -c 0
  timeout --signal=TERM --kill-after=5s 45s "$engine" \
    --bench --native "$model" --max-context 8192 --parallel 1 \
    --kv fp16 --vram-reserve-mib 2048 --ram-experts-gb 3 \
    --draft off --prompt-len 128 --gen 8
)
```

### Reproduction status

Reproduced locally

### Strata version

Terminal 0.2.19, `strata-inkling` 0.1.65, Linux x86_64 CUDA 13 package, engine commit `c6f6427d9de9`.

### Graphics card, memory and system

One NVIDIA GeForce RTX 4070 SUPER (12,282 MiB reported), driver 615.78.08, AMD Ryzen 7 7800X3D, 61 GiB usable RAM, Omarchy 4.5.0, Linux `7.2.15-1-omarchy` x86_64.

### Model, size and settings

Inkling-2-Mini SQ-10, 8K context, one request, vision off. The original serving run used FP8 KV, DSpark, a 2,048 MiB VRAM reserve, and Strata-generated expert budgets of 8.60 GB VRAM and 0.00 GB RAM. The engine reported 2,339 experts in VRAM and 733 on the host.

### Logs

Standalone benchmark output:

```text
[inkling2-ep] Loaded strata (SQ-10) | gpus=1 (p2p=pinned, ep_split=0.50) | experts: 2312 VRAM (GPU0=6.16 GiB, GPU1=0.00 GiB), 760 host (doorbell=off, numa=1, thr=0) | kv=fp16
timeout: the monitored command dumped core
```

The original serving crash recorded `SIGSEGV` (`SEGV_MAPERR`). Top frames from `coredumpctl`, with names demangled and shortened:

```text
strata::quant::sk::gemm_sk_q8_1
strata::inkling2::eval_inkling2_expert_batch
EpDualGpuCudaBackend::forward_batch
ink2::run_serve_loop
```

The kernel logged `segfault at 900` in `strata-inkling`. No OOM kill appeared in the kernel journal around the serving crash. `strata doctor --live --profile launch-inkling-sq10-8k` found the engine, model and tokenizer files and reported `Live GPU probe: ok`.

Related model support discussion: #6442.

関連リンク

インストール・モデル・リリースへのサイト内リンク。