#2678 [sm_120 / RTX 6000D] native Q5_K head dequant crashes on long requests: "unspecified launch failure"

说明

# GitHub Issue: native Q5_K head dequant crashes on long requests — "unspecified launch failure" on RTX PRO 6000D (Blackwell, sm_120)

> Copy-paste ready. Paste into https://github.com/Niko1221/Strata/issues/new

---

## Summary

On an **RTX PRO 6000D** (GB202, Blackwell 2.0, compute capability **12.0 / sm_120**), the **native Q5_K output head** dequant kernel crashes with:

```
dequant launch: unspecified launch failure
```

on **long requests**. Short requests (≤ ~10 tokens) work fine; anything from ~128 tokens up crashes the engine every time. The crash is tied to the native head dequant kernel (`native_q5_k_f32`), and it is **worse on newer engine versions** — an original (non-abliterated) native IQ pack that ran long requests cleanly on **v0.1.40.2** for days starts crashing at just **128 tokens on v0.1.44.1**.

This looks like the same class of architecture-guard bug already seen in your `#371` (qsa_prompt_attn_batch on sm_70): a kernel guard that checks only the compute-major version, so a kernel that is wrong for the actual device gets launched, and only fails on the larger grid / higher-memory path that a long request triggers.

## Environment

- **GPU:** NVIDIA RTX PRO 6000D (Blackwell 2.0, GB202, compute capability 12.0)
- **CUDA Toolkit:** 13.3
- **OS:** Ubuntu 24.04
- **Pack:** native IQ pack — Qwen3.8-Flash-Next-GSQ-RCO-IQ3_S (IQ3_S), loaded with `--native SHARD1` (required; the pack refuses to start without it)
- **PLE:** `--ple-gguf` provided
- **Engine versions tested:** v0.1.40.2 (original pack runs cleanly), v0.1.44.1 (crashes)
- **Config flags:** `--expert-cache auto --prefill auto:16384 --spec 4 --spec-min-p 0.5 --mtp <mtp>/rt --max-context 262144 --kv int8 --kv-resident 32768 --batch 6 --batch-mtp --vram-reserve-mib 15000`

## Reproduction

A native IQ pack with `--native` loads the native head (log line: `experimental native Q5_K head, 521472000 bytes, in 0.1 s`) and then `session is up`. After that:

- short prompt → OK
- long prompt (≥ ~128 tokens) → engine dies, last log line: `dequant launch: unspecified launch failure`

The engine exits with status 1; the next request restarts it.

## Test data (RTX PRO 6000D, native IQ IQ3_S pack, `--native`)

### Engine **v0.1.40.2** — original (non-abliterated) IQ3_S pack — all requests OK

| Request | gen tokens | decode speed (tok/s) | prompt speed (tok/s) | HTTP |
|---|---|---|---|---|
| short (10) | 10 | 99.8 | 246 | 200 |
| medium (128) | 128 | 134.3 | 266 | 200 |
| long (512) | 512 | 151.0 | 559 | 200 |
| very long (1024) | 1024 | 161.1 | 553 | 200 |

### Engine **v0.1.44.1** — same original IQ3_S pack — crashes from medium up

| Request | gen tokens | decode speed (tok/s) | HTTP |
|---|---|---|---|
| short (10) | 10 | 125.9 | 200 |
| medium (128) | — | — | **503** |
| long (512) | — | — | **503** |
| very long (1024) | — | — | **503** |

So v0.1.44.1 is a **regression** relative to v0.1.40.2 on this card: the native head dequant that used to survive long requests now fails at 128 tokens.

## Why I think it's an architecture-guard / sm_120 issue

1. Your own history of `unspecified launch failure` issues is dominated by **architecture-guard bugs** — e.g. `#371` (qsa_prompt_attn_batch on sm_70: the host guard compared only the major version 7 while the kernel needs sm_75, so a Turing kernel was launched on a V100 and hit `__trap()`). The native Q5_K head dequant (`native_q5_k_f32`, `native_head.cpp`) is a dedicated CUDA kernel; if its launch guard similarly checks only the major compute version, sm_120 could slip through and run a kernel whose PTX/instructions are not valid on Blackwell.
2. llama.cpp upstream has the same class of problem on Blackwell: `ggml-org/llama.cpp#19662` reports MXFP4 template instances failing to assemble for sm_120 with `"Instruction 'mma with block scale' not supported"`, and MMQ/dequant kernels crashing on sm_120 until built with a correct `-DCMAKE_CUDA_ARCHITECTURES=120` and a matching CUDA toolkit.
3. The **short-request OK / long-request-crash** pattern matches a kernel that is fine at small grid / low shared-memory but trips a hardware limit or unsupported instruction only on the larger launch path that a long context requires.

## Suggested fixes to look into

- Check whether the native Q5_K head dequant launch guard consults the **minor** compute version (like the `#371` fix did) and whether sm_120 needs a dedicated kernel path or a fallback to the canonical head.
- Verify the native head kernel is compiled for / validated on sm_120 (CUDA arch 120), same care as the Blackwell MMQ/dequant build guidance in `#19662`.
- Consider whether the native head can be disabled or routed to a portable path when the guard detects sm_120, so the canonical `output.weight` path is used instead.

## Attachments

Full engine log tail (startup → `session is up (engine 0.1.44.1)` → crash):

```
strata generate: experimental native Q5_K head, 521472000 bytes, in 0.1 s
strata generate: GPU 0: NVIDIA RTX 6000D, compute capability 12.0
...
strata generate: session is up (engine 0.1.44.1)
...
strata serve: 14571 MiB of VRAM free with everything loaded
dequant launch: unspecified launch failure
```