问题 / #3626

#3626 [Engine] Coder IQ1_M: SIGSEGV in kernels::cpu::native_gu_rows (ExpertPool), deterministic across 0.2.0-beta … 0.2.2 (RTX 4070)

open · @saschamarco · 0 评论 · 去 GitHub 看

Server & APIAMD / HIPNVIDIA / CUDAModels & quantsSecurityLinux

说明

## What happens

Serving `Qwen3.8-Flash-Next-GSQ-RCO-IQ1_M` (coder pack) segfaults the engine. It
crashes reliably on the first generation requests of a session, never at load time
(load always reports `session is up`, `slot 0 verified`). The same machine serves
IQ3_S all day without a single crash.

## Environment

- Linux (Manjaro, kernel 7.2.11), AMD Ryzen 7 5700G (Zen 3, 8 physical / 16 logical), 62 GiB RAM, 127 GiB swap
- GPU 0: RTX 4070 12 GB (`gpu_roles: 0=main, 1=off`), GPU 1: RTX 3050 (unused by this config)
- CPU ISA as exposed by ggml-cpu autotune: AVX2 + BMI2 available, no AVX-512
- Engine 0.1.47 (Strata 0.2.2); also reproduced on 0.1.x from 0.2.0-beta (19.10) and 0.2.1 (20.10)
- Model: `Qwen3.8-Flash-Next-GSQ-RCO-IQ1_M-0000{1,2}-of-00002.gguf`, pack `coder-iq1_m`,
  `expert-profile-coder.bin` (12288 pairs)

## Reproduce

```
engine/strata --serve --pack packs/coder-iq1_m \
  --native models/coder-IQ1_M/Qwen3.8-Flash-Next-GSQ-RCO-IQ1_M-00001-of-00002.gguf \
  --ple-gguf models/coder-IQ1_M/Qwen3.8-Flash-Next-GSQ-RCO-IQ1_M-00002-of-00002.gguf \
  --expert-profile data/expert-profile-coder.bin --expert-cache auto --prefill auto \
  --spec 4 --spec-min-p 0.5 --mtp mtp/rt --max-context 204800 --kv q4_0 --kv-resident 32768
```

Send any chat request. Today (21.10.) four sessions all SIGSEGV'd within the first
request each (16:36, 16:38, 16:48, 16:53). Historical coredumps show the same
signature on 0.2.0-beta (`--max-context 262144 --kv int8`) and 0.2.1 (with and
without cvec flags), 19.–20.10.

## Crash signature (identical in all dumps)

Main thread (verifier path), one of the pool threads:

```
#0 strata::kernels::cpu::native_gu_rows(NativeFmt const&, unsigned char const*, unsigned char const**, int, float const*, int, int)   (strata + 0x80df47)
#1 strata::kernels::cpu::ExpertPool::drain(int, ExpertScratch&, unsigned)                                                               (strata + 0x7ec7df)
#2 strata::kernels::cpu::ExpertPool::run_phase(int, int)                                                                                (strata + 0x7edab6)
#3 strata::kernels::cpu::ExpertPool::run_split_multi_native(NativeFmt const&, ExpertJobMulti*, int)                                      (strata + 0x7edbc1)
#4 strata::core::expert_pool_dispatch_multi(...)                                                                                        (strata + 0x1c1035)
#5 (anonymous)::drive_pool_multi(...)                                                                                                   (strata + 0xcbc6c)
#6 strata::core::Verifier::run(...)                                                                                                     (strata + 0x2237bf)
```

The seven pool workers sit in `ExpertPool::worker → drain → native_gu_rows` at the
same instruction. The log ends right after
`strata verify: captured the 4-token window (upload no error, sync no error)`.

## What I ruled out

- **Not VRAM pressure**: the run logs the <512 MiB warning (#2065), but the config
  crashes the same way with more free VRAM, and Coder IQ1_M is benchmarked stable on
  an RTX 4070 12 GB (#3452, #3249).
- **Not a corrupt/truncated pack or GGUF**: byte sizes match upstream exactly; I
  parsed all 48 `native_experts.txt` rows programmatically — every `gate_off/up_off/down_off +
  blob_bytes` lies inside shard 1 and matches the IQ1_M block size.
- **Not quant-width per se**: the same engine args on IQ3_S (same card, same host)
  never crash — the difference is the IQ1_M quant / coder pack.

## Coredump

A full dump of the last crash (25.6 GB, zstd) is kept:
`core.strata-host.1000.f8eff642fa2243969edabf71852762a0.342032.1792594314000000.zst`.
Happy to upload it or run `gdb` probes (faulting address, `$pc`) if you tell me
what is most useful.

## Isolation I can run on request

- `STRATA_PREFILL_CPU_SHARE=0` (the crash is in the CPU expert path)
- `STRATA_ROUTE_TAIL_SKIP=0`
- `--spec 0` to confirm the verify path is the trigger

The stack points into the native (IQ1_M) row reader — my guess is an out-of-range
row pointer / a row one superblock shorter than the size the pack table promises,
which load-time `slot 0 verified` would not catch.

本站相关内容

相关页面的快捷入口。