Issues / #2065
#2065 Engine hangs at layer 48 during decode when VRAM headroom drops below ~500 MiB
open · @bang-imvt · 1 Kommentare · Auf GitHub
BenchmarksServer & APINVIDIA / CUDAModels & quants
Beschreibung
## Summary
Engine reproducibly hangs during decode with `--batch 2` + 262144 context on a
single RTX 4090 D (24 GB). Two occurrences on 2026-10-12, both caught by the #29
watchdog ("no progress for 60 s during a request (decode -1)"), both leaving
thread-stack dumps.
## Environment
- Engine: 0.1.41
- GPU: RTX 4090 D 24 GB (driver 591.74)
- RAM: 256 GB
- Model: Qwen3.8-Flash-Next IQ3_S
- Command:
strata.exe --serve --pack ... --expert-cache auto --prefill auto:8192
--spec 4 --spec-min-p 0.5 --mtp ... --max-context 262144 --kv int8
--kv-resident 32768 --vram-reserve-mib 1021 --batch-mtp --batch 2
## Symptoms
Both stalls are identical:
- stage "decode -1" for 60 s
- host idle for ~60 s
- expert pool: 288/288 jobs done, 95/95 workers sleeping
- verify window: 4 tokens, host at layer step 48; the GPU rang 48
- memory: ~34-55 GiB resident, ~108 GiB RAM available (RAM is NOT the issue)
## Key finding: VRAM exhaustion
Startup leaves very little headroom:
strata serve: 538 MiB of VRAM free with everything loaded
strata verify: capturing the 4-token window (500 MiB of VRAM free)
strata verify: capturing the 1-token window (477 MiB of VRAM free)
strata verify: capturing the 3-token window (462 MiB of VRAM free)
strata verify: capturing the 2-token window (422 MiB of VRAM free)
strata verify: capturing the 6-token window (413 MiB of VRAM free)
strata verify: capturing the 5-token window (395 MiB of VRAM free)
Decode speed collapses right before each hang:
60,902 tok prompt -> 82.0 tok/s
66,375 tok prompt -> 90.9 tok/s
70,118 tok prompt -> 3.3 tok/s <- collapse
70,494 tok prompt -> 6.0 tok/s <- then hang
## Dumps
- strata-stall-64352.dmp (795 KB, 09:43)
- strata-stall-60340.dmp (791 KB, 14:48)
(I can attach both.)
## Suspected cause
VRAM headroom hitting ~400 MiB prevents the GPU from completing a kernel at
layer 48; the host and CPU expert pool are both fully idle while waiting.
## Question
Would you recommend raising --vram-reserve-mib and/or dropping to --batch 1, or
is there a known issue with layer 48 under low VRAM?
[strata-stall-60340.dmp](https://github.com/user-attachments/files/33325088/strata-stall-60340.dmp)
[strata-stall-64352.dmp](https://github.com/user-attachments/files/33325089/strata-stall-64352.dmp)Mehr auf der Site
Links zu Install, Modellen, Releases.