Issues / #890
#890 HIP: "parallel": 2 + --layer-split on 2x R9700 exits (code 1) after "captured the batch window over slots 0,1"
open · @asaffulks · 6 コメント · GitHub で見る
BenchmarksSetup & installMulti-GPUAMD / HIPNVIDIA / CUDAModels & quants
本文
**Setup** - Strata v0.1.39 (6f32ec0) built from source, HIP backend, gfx1201, TheRock ROCm 7.10 wheels, Ubuntu 26.04, kernel 7.0 - 2x Radeon AI PRO R9700 (32 GB), Qwen3.8-Flash-Next IQ3_S - `layer_split "24"`, `--trim-stage-weights --mmap-experts`, MTP `--spec 4` — 100% of experts resident in VRAM **What happens** With `"parallel": 2`, the first time two requests decode together the engine exits with code 1 and no error message. Last log line: ``` strata verify: captured the batch window over slots 0,1 ``` `--batch-groups 2` crashes the same way (last line `captured the batch window over slots 0`). **What works** - Same box single-lane: ~1,227 tok/s prompt (29K tokens), 52-64 tok/s decode, stable. - `"parallel": 2` on a single R9700 (no split) — fine. - `"parallel": 2` with a layer split on 3x RTX 5070 Ti (CUDA) — fine. So it looks specific to batch decode + layer split on HIP. Happy to run a debug build or send the full log.
関連リンク
インストール・モデル・リリースへの站内リンク。