Issues / #2403
#2403 --kv q4_0 faults on the first request when the KV streams (--kv-resident < --max-context) and MTP drafts: "checkpoint save: conversation snapshot running-state copy: unspecified launch failure" (0.1.43.1, RX 6900 XT gfx1030, Windows)
open · @ByteJuggler · 1 Kommentare · Auf GitHub
Setup & installServer & APIAMD / HIPNVIDIA / CUDAModels & quantsSecurityWindows
Beschreibung
## What happens
With `--kv q4_0`, the engine loads normally, but every request fails at once. The engine log says:
```
strata verify: captured the 6-token window (upload no error, sync no error)
strata verify: captured the 3-token window (upload no error, sync no error)
strata serve: checkpoint save: conversation snapshot running-state copy: unspecified launch failure
```
The client gets `saving a conversation checkpoint failed`, and the engine exits with code 1. The server restarts it,
and the next request fails the same way.
The checkpoint copy is only where the fault shows up. With `AMD_SERIALIZE_KERNEL=3 AMD_SERIALIZE_COPY=3
HIP_LAUNCH_BLOCKING=1`, the error comes from the first verify window instead:
```
strata verify: captured the 6-token window (upload no error, sync no error)
strata serve: verify: launch: unspecified launch failure
```
That is the `cudaGraphLaunch` in `src/core/verify.cpp`, so a kernel inside the captured verify graph faults.
## What it needs
One request ("Name three rivers.", 64 tokens, no thinking) per config. Each config changes one thing from the failing one:
| config | result |
|---|---|
| failing config (below): `--kv q4_0`, `--max-context 65536 --kv-resident 32768`, `--spec 4 --mtp ...` | **fault** |
| same, default KV (int8, no `--kv`) | ok |
| same, `--kv-resident 65536` (no streaming) | ok |
| same, `--max-context 8192 --kv-resident 8192` | ok |
| same, `--spec 2`, no `--mtp` | ok |
| same, `STRATA_HIP_PROMPT_F16` unset | **fault** |
| same, no control vector (`--control-vector-scaled` etc.) | **fault** |
| same, kernels serialized (env above) | **fault** (at `verify: launch`) |
| same, `STRATA_NO_BATCH_KV_STEP=1` | **fault** |
| same, `STRATA_NO_MULTI_GR=1` | **fault** |
So the fault needs all three: q4_0 KV, KV streaming (`--kv-resident` below `--max-context`), and MTP drafting.
`--kv k8v4` with the same streaming and MTP setup runs without errors.
With `STRATA_VERIFY_NODES=1`, the 6-token verify graph includes 12 x `kv_append_q4_kernel` (which takes the
streaming `KvHostPools`), 12 x the KV-stream `resolve_kernel` / `copy_kernel`, and 12 x `attn_chunk_kernel_pre75`.
The q4_0 append into the host pool, or the stream map, are my first guesses, since the fault needs q4_0 and streaming
together. I have not confirmed which kernel faults. On Windows the driver reports the fault as a "driver timeout"
(the AMD Bug Report Tool pops up).
## Failing command
```
strata.exe --serve --pack <packs>\q2_0 --native Qwen3.8-Flash-Next-GSQ-RCO-Q2_0-00001-of-00002.gguf
--ple-gguf Qwen3.8-Flash-Next-GSQ-RCO-Q2_0-00002-of-00002.gguf --expert-profile data\expert-profile.bin
--expert-cache auto --prefill auto --spec 4 --spec-min-p 0.5 --mtp <mtp>\rt --max-context 65536
--kv-resident 32768 --control-vector-scaled <...>speed-projection.gguf:1.0 --control-vector-layer-range 4 44
--cvec-mode project --cvec-dir per-layer --kv q4_0
```
Env: `STRATA_HIP_PROMPT_F16=1` (it fails without it too).
Log lines that may matter:
```
strata generate: KV streaming: 32768 of 65536 cells per QSA layer in VRAM, the K/V in 0.42 GiB of pinned RAM
strata generate: expert cache 5585 slots, 7.19 GiB of VRAM
strata verify: window up to 6 tokens, 55.2 MiB of device buffers
strata serve: 2247 MiB of VRAM free with everything loaded
```
## System
- Strata 0.1.43.1, prebuilt Windows HIP zip (ROCm 10.2.0a20260930)
- AMD Radeon RX 6900 XT 16 GB (gfx1030 / RDNA2), AMD Software 26.8.1 (driver 32.0.21045.5002)
- Intel i7-5930K (6 cores, AVX2, no AVX-512), 64 GB RAM, Windows 11 Pro 23H2 (22631)
- Model Q2_0 (GSQ-RCO, native pack)
Possibly related: #1932 (a device fault that also first shows up at checkpoint save, on gfx1100).
I can run more variants or a debug build if that helps.
Mehr auf der Site
Links zu Install, Modellen, Releases.