Issues / #2694
#2694 [Bug]: Metal (M1 Ultra): engine stalls at token 0 reading an 8.7K-token prompt — "no progress for 60 s", exit -6 (0.1.44.1, stock config)
open · @beettlle · 1 Kommentare · Auf GitHub
BenchmarksSetup & installServer & APIModels & quantsWindows
Beschreibung
### What happened On Apple Silicon (Metal backend), the engine stalls at **token 0** while reading a longer prompt (8,722 tokens). After 60 s the watchdog stops it with exit code -6 and asks for a report under #29. A short prompt (58 tokens) on the same install completes, but slowly: prefill runs at 1.6 tok/s and decode at 2.3 tok/s. This happened on 2 of the 4 engine starts in the log, once with `--max-context 32768` and once with `--max-context 65536`. Everything else is the stock `setup.sh` config (`--expert-cache 64`, `--prefill auto`, `--mmap-experts`, `STRATA_METAL_STREAM=1`). This is a different bug from #2671: no flags were changed, and the engine stalls instead of exiting with code 1. **Steps** 1. Clean `./setup.sh` install of v0.1.44.1 (commit `296b59d`), Qwen3.8-Flash-Next GSQ-RCO IQ3_S. 2. `./run-iq3_s.sh`, then wait for `ready: http://127.0.0.1:8081/v1`. 3. Send a chat request with a prompt of about 8.7K tokens. **Console output** ``` ready: http://127.0.0.1:8081/v1 (OpenAI: /v1/chat/completions, Anthropic: /v1/messages, context 65536 tokens) [strata] reading the prompt: 8,722 tokens, 10 s so far ... [strata] reading the prompt: 8,722 tokens, 61 s so far [strata] the engine stopped unexpectedly (exit code -6). The engine stopped itself because it had stopped making progress - a hang it caught. Its log line: strata serve: no progress for 60 s during a request (reading the prompt, from token 0) - stopping the engine so the server starts it again (issue #29) ... [strata] done: 0 tokens in 62 s (0.0 tok/s) (error, cancel=False) ``` **Stall report** ``` strata serve: stall report (engine 0.1.44.1-metal): stage "reading the prompt, from token 0" for 59 s; 0 layers served since the last finished step (0 = stopped, more = slow) VRAM free when this request began: 42037 MiB expert pool: epoch 0, batch epoch 0: 0 of 0 jobs claimed, 0 done; 19 of 19 workers parked, 19 sleeping; mode 0 expert pool threads: w0..w18=sleeping; host idle for 112309 ms verify window (last window, not the current stage): 0 tokens at position 0, host at layer step 1; the GPU rang 0; flags: served 0, plan (A) 0, copies (B) 0 memory: 0 MiB resident, 0 MiB in swap, 0 MiB RAM available; 0 major page faults so far strata: released the verify window's GPU waits (#267): the GPU finished in 0 ms ``` **What the log shows** - No work was dispatched: 0 layers served, 0 expert jobs, every pool worker asleep, and the GPU never rang. This looks like a hang rather than slow progress. - `host idle for 112309 ms` is longer than the 60 s watchdog window. The host thread was already idle before this request started. - No major page faults, so the engine wasn't waiting on the disk. - The memory line reads 0 for everything. At startup the log also says `memory guard: the available RAM cannot be read; parking and session save/restore are refused`. On macOS the stall report's memory figures may never be filled in. - `the arena is small: the prompt's 1.63 GB of buffers are their own allocation, not a region of its slots`, and `the prompt path allocates its own buffers (too few cache slots to borrow)`. - `PCIe probe failed -> pcie_frac default 0.55`. This is expected on unified memory, but it shows up on every start. **Setup notes** - The model files are on an external **USB** SSD (`/Volumes/NVMe`, APFS). The expert arena fill reads at about 0.90 GB/s. - No `strata-stall-*.dmp` file was written. That dump is Windows-only, as far as I can tell. I'm happy to run a debug build, try environment variables (for example `STRATA_PREFILL_CPU_SHARE=0` or `STRATA_METAL_WIRE_ARENA=0`), or test a patch on this machine. ### Strata version, GPU, OS 0.1.44.1-metal (commit 296b59d) · Apple M1 Ultra, 64 GB unified memory · macOS 27.0.1 (26A434) ### Engine log (attach `strata-iq3_s.log`)
Mehr auf der Site
Links zu Install, Modellen, Releases.