反馈 / #1971

#1971 0.1.42: the engine stops in "reading the prompt (batched)" right after the conversation cache's reused prefix (about one run in two, --vram-reserve-mib 1000, RTX 4070 SUPER 12 GB)

open · @LeGeRyChEeSe · 0 评论 · 去 GitHub 看

BenchmarksSetup & installServer & APIAMD / HIPNVIDIA / CUDAModels & quantsWindowsLinux

说明

**Summary**

On 0.1.42 the hang guard (#29) stops the engine during a prompt read: "no progress for 60 s during a request (reading
the prompt (batched): layer 1 of the prompt chunk from token N)". The engine then exits with 3221226505 (0xC0000409),
as in #1865. N is always the end of the previous turn's context, i.e. the first chunk read after the conversation
cache's reused prefix. The turn before the stall already reads its prompt about twice as slowly as in a run that does
not stall (837 tok/s against 1,820). It happens with and without `--vision`, and with and without the route tail skip.
It never happened on 0.1.41.

**How often** (a replay of coding-agent turns, one engine process per run, `--vram-reserve-mib 1000`):

| engine | runs | stalled |
|---|---|---|
| 0.1.42, `--vision` | 7 | 4 |
| 0.1.42, no vision | 4 | 2 |
| 0.1.41, no vision | 2 | 0 |
| 0.1.42, `--vision`, `--vram-reserve-mib 450` | 2 | 0 |

The stall is always at the same place for a given corpus: turn 3 at token 15866 (turn 2 ended at 15871), or turn 4 at
token 43718 (turn 3 ended at 43723).

**Setup**
- RTX 4070 SUPER 12 GB (PCIe 4.0 x16, driver 617.14, CUDA 13.1), Ryzen 5 5600X, 64 GB DDR4-3600, Windows 11 Pro build
  26300, 30 GB page file. "Windows commit: 24.3 GiB left of a 93.9 GiB limit" at start; about 7 GB of RAM available at
  the stall.
- Engine 0.1.42 built from the tag (CUDA archs 86;89). Flash-Next IQ2_XS (GSQ-RCO), the expert profile the repository
  ships.
- Args: `--expert-cache auto --prefill auto --spec 4 --spec-min-p 0.7 --mtp <rt> --max-context 131072 --kv int8
  --kv-resident 32768 --pcie-frac 0.55 --vram-reserve-mib 1000` (+ `--vision` in the vision runs). 3,585 cache slots
  without vision, 2,934 with it.
- Load: 23 coding-agent turns (2.5K-103K tokens) from 5 simulated agent sessions on this repository at a fixed commit
  (the tool results are the repository's real files), replayed in order, so a turn shares its session's prefix with the
  one before and the conversation cache reuses it; 256 tokens at T = 1, one request at a time.

**Stall report** (no vision run):
```
strata serve: prompt 15871 tokens = 5062 reused + 10809 read in 12909 ms (837.3 tok/s), 52 generated in 586 ms (88.8 tok/s), drafts accepted 35 of 42, 2 checkpoints
strata serve: no progress for 60 s during a request (reading the prompt (batched): layer 1 of the prompt chunk from token 15866) - stopping the engine so the server starts it again (issue #29)
strata serve: stall report (engine 0.1.42): stage "reading the prompt (batched): layer 1 of the prompt chunk from token 15866" for 61 s; 0 layers served since the last finished step (0 = stopped, more = slow)
  expert pool: epoch 5204, batch epoch 5204: 18 of 18 jobs claimed, 18 done; 5 of 5 workers parked, 5 sleeping; mode 0
  expert pool threads: w0=sleeping w1=sleeping w2=sleeping w3=sleeping w4=sleeping; host idle for 61931 ms
  verify window (last window, not the current stage): 4 tokens at position 15920, host at layer step 48; the GPU rang 48; flags: served 48, plan (A) 48, copies (B) 48
  memory: 36205 MiB resident, 49383 MiB committed, 7092 MiB RAM available; 19156906 page faults so far
```
In runs that do not stall, the same turn 2 reads its 10,809 tokens in 5.9 s (1,820 tok/s).

**Not checked**
- Linux; another card; another quant.
- Whether 450 is truly free of it, or only rarer (0 of 2 runs).
- Odd detail, maybe chance: the same 0.1.42 engine driven by the 0.1.39 `serve/server.py` (the same 15,866
  tokens reused at turn 3) did not stall in 4 runs at 1000.

The thread-stack dumps (`strata-stall-*.dmp`, 6 of them) and the full engine logs are available: tell me which ones help.

本站相关内容

相关页面的快捷入口。