Issues / #890

#890 HIP: "parallel": 2 + --layer-split on 2x R9700 exits (code 1) after "captured the batch window over slots 0,1"

open · @asaffulks · 6 commentaires · Sur GitHub

BenchmarksSetup & installMulti-GPUAMD / HIPNVIDIA / CUDAModels & quants

Description

**Setup**
- Strata v0.1.39 (6f32ec0) built from source, HIP backend, gfx1201, TheRock ROCm 7.10 wheels, Ubuntu 26.04, kernel 7.0
- 2x Radeon AI PRO R9700 (32 GB), Qwen3.8-Flash-Next IQ3_S
- `layer_split "24"`, `--trim-stage-weights --mmap-experts`, MTP `--spec 4` — 100% of experts resident in VRAM

**What happens**
With `"parallel": 2`, the first time two requests decode together the engine exits with code 1 and no error message. Last log line:
```
strata verify: captured the batch window over slots 0,1
```
`--batch-groups 2` crashes the same way (last line `captured the batch window over slots 0`).

**What works**
- Same box single-lane: ~1,227 tok/s prompt (29K tokens), 52-64 tok/s decode, stable.
- `"parallel": 2` on a single R9700 (no split) — fine.
- `"parallel": 2` with a layer split on 3x RTX 5070 Ti (CUDA) — fine.

So it looks specific to batch decode + layer split on HIP. Happy to run a debug build or send the full log.

Sur le site

Liens install, modèles, releases.