Issues / #2438
#2438 Layer split on an unequal NVIDIA pair (RTX 5080 + RTX 4070 Ti) hard-crashes the machine at 240K context; dump write fails (Windows 11, v0.1.42)
open · @leonininder · 1 comentários · No GitHub
BenchmarksSetup & installMulti-GPUNVIDIA / CUDAModels & quantsWindows
Descrição
## Body **Setup** - Strata v0.1.42 release (`strata-windows-x64-cuda12.zip`, BUILD.json: CUDA 12.9, archs 60–89 + PTX), Windows 11 Home, driver 617.42. - Unequal pair: RTX 5080 16 GB (sm_120 — runs this build's sm_89 **PTX**, no sm_120 cubin in the zip) + RTX 4070 Ti 12 GB (sm_89). PCIe probe 6.4 GB/s. - Model: Qwen3.8-Flash-Next abliterated IQ3_XXS, 262,144 context, `--kv int8 --kv-resident 32768`, `--spec 3` + `mtp/rt`, `--expert-cache auto`, `--vram-reserve-mib 1024`. - Layer split via `--layer-split auto --split-device 1`, `--vram-reserve-later-mib 512` (5080 first; auto picked `layers 0-34 / 35-47`). PCIe frac auto-probed to 0.18. **What happened** Three consecutive ~238K-token prompt runs completed (log: `prompt 237976 tokens = ... 94253 ms (2524.9 tok/s)`, KV streaming 82–84% VRAM hits, expert reads over PCIe reported per request), then the machine **hard-crashed** (Kernel-Power 41). The crash dump write itself failed: `volmgr` event 161, `BugCheckProgress 0x00020020` — no minidump, so unfortunately no bugcheck code. The same 240K ×3 workload is stable on the same engine in single-GPU mode (`--gpu` only), and the split runs were fine at 8K/70K. **Hypothesis** This build has no sm_120 cubin (BUILD.json archs stop at 89, `ptx: true`), so the 5080 executes PTX-JIT kernels. The layer-split + long-context KV-streaming path exercises cross-card hand-off per verify window; a JIT edge case under that pressure is my first suspect (second: the known asymmetric-pair rough edges in #2052/#2239). Kernel dump capture is now enabled here, so a repeat run should produce a dump — happy to run a bisect-style probe if you point me at the right knobs (e.g. `STRATA_SPLIT_RING`, `--kv-grow`, forcing precompiled cubins). **Also measured (for the TEST_REQUESTS unequal-pair ask)** Full report with per-run data ready for a `bench/results/` PR (both card orders + helper-cache arm): single-GPU won every tier; both split orders lost ~30-55% prefill and ~12% decode vs single 5080; helper cache (`--expert-cache-device1 5000 --remote-expert-opt`, 3,306 experts on the 4070 Ti) collapsed decode to 23.6 tok/s (spread 16.6–48.5) vs 85.0 control. `cuDeviceCanAccessPeer` = 0 both ways (WDDM GeForce), so peer-mapped designs are untestable on this host. *(Bench report: PR pending — link will follow.)*
No site
Links install, modelos, releases.