Issues / #2313

#2313 [Bug]: --calibrate on Linux/HIP (RX 7800 XT eGPU over Thunderbolt): engine stops unexpectedly (exit code -15) with no ERR line

open · @RandallSP · 0 comments · View on GitHub

BenchmarksSetup & installServer & APIAMD / HIPNVIDIA / CUDAModels & quantsWindowsLinux

Description

**Environment**

- Strata 0.1.43.1, engine 0.1.43.1, commit `a0116b27` (which was `origin/main` at the time).
- HIP backend on an RX 7800 XT (gfx1101, eGPU over Thunderbolt) beside an RTX A2000; CachyOS Linux, kernel 7.2.9-2-cachyos; 124 GB RAM.

**What happened**

`--calibrate` aborts with the engine gone (exit code -15), on a Linux + HIP setup.

1. `./setup.sh --calibrate` (config `strata-iq3_s.json`, engine 0.1.43.1, commit a0116b27, which was `origin/main` at the time).
2. The PCIe-share sweep, the draft-floor sweep, and the 7- and 5-worker stages all completed and printed their medians.
3. During the 4-worker stage the engine started (its log banner: `2026-10-13 10:57:48 engine started`), reached `session is up`, and served requests.
4. At about 11:00:5x the engine died. setup printed:

```
  [!]  the tuning did not finish (the engine stopped unexpectedly (exit code -15)): the default settings stay
       the engine said: strata serve: suffix drafts: 1 windows, 1 of 1 drafts accepted
Exception ignored while finalizing file <_io.TextIOWrapper name=8 encoding='utf-8'>:
Traceback (most recent call last):
  File "/mnt/NX/Strata/setup.py", line 4204, in calibrate_config
    return False
BrokenPipeError: [Errno 32] Broken pipe

  [!]  this PC is NOT tuned: the tuning failed (the reason is above); the model starts with the default settings
```

Expected: the tuning finishes, or a start that fails inside a later stage drops that candidate (#1337) instead of ending the whole run.

**What I checked** (no supervisor here, no llama-swap; the engine was started by setup itself)

- Engine log: no `ERR verify` line that day and no `issue #29` line (the only one in the file is from 2026-10-11).
- Kernel: `ring .* timeout` 0, `device wedged` 0, `reset compute queue` 0, `amdgpu_pmops_suspend` 0, no devcoredump entry. The eGPU stayed present (`runtime_status=active`, link 16 GT/s x16).
- VRAM came back within a minute: `mem_info_vram_used` 16,959,606,784 B -> 191,897,600 B.
- Code reading: `-15` is a SIGTERM. `setup.py` and `tools/calibrate.py` cannot send one: their stop path is `QUIT` then `proc.kill()` (SIGKILL, -9), and the READY timeout (`ENGINE_READY_S`, 900 s) also kills with SIGKILL and reports a different message. Only `serve/server.py` sends SIGTERM to the engine: `_pump` on a `FATAL_PREFIXES` line, and `close()` when `QUIT` does not end the engine in 20 s. Neither trace is in the log.
- So the sender of the SIGTERM is unknown from here. The message text matches #2257, where the SIGTERM came from outside (a supervisor signalling the whole process group).

**Engine log tail** (`strata-iq3_s.log`, the 4-worker engine, last 20 lines)

```
strata generate: session is up (engine 0.1.43.1)
strata generate: token graph hit path: 4838 resident experts, decided on the device
strata serve: prompt chunk auto: 8192 tokens, a 384-slot ring
strata serve: the prompt path borrows 2210 CUDA0 cache slots (4.18 GiB)
strata hc: CUDA0: the hyper-connection read runs as staged (the norm per token and stream, the down projection's activations staged ahead by cp.async); checked bit for bit against the plain read on this card (STRATA_HC_SPLIT=1 or 0 for the earlier ones)
strata verify: window up to 6 tokens, 74.7 MiB of device buffers
strata mtp: draft head over 106299 tokens (212.9 MiB)
strata serve: adaptive overlap: off (STRATA_ADAPT_OVERLAP=1 turns it on)
strata serve: 614 MiB of VRAM free with everything loaded
strata verify: capturing the 4-token window (519 MiB of VRAM free)
strata verify: captured the 4-token window (upload no error, sync no error)
strata verify: capturing the 1-token window (489 MiB of VRAM free)
strata verify: captured the 1-token window (upload no error, sync no error)
strata verify: capturing the 2-token window (487 MiB of VRAM free)
strata verify: captured the 2-token window (upload no error, sync no error)
strata verify: capturing the 3-token window (487 MiB of VRAM free)
strata verify: captured the 3-token window (upload no error, sync no error)
strata serve: prompt 774 tokens = 0 reused + 774 read in 16216 ms (47.7 tok/s), 111 generated in 5090 ms (21.8 tok/s), drafts accepted 57 of 110, 1 checkpoints
strata serve: decode expert cache hit rate: 61.5% (48678 hits / 79158 lookups); 522 more read by the GPU over PCIe or from another GPU (0.7% of all 79680 routed)
strata serve: KV streaming: 99.37% of 424044 block reads hit VRAM, 10.7 MiB read from RAM
```

The log ends there (file mtime 11:05:12). No error line, no QUIT line, nothing after it.

**Config**: `strata-iq3_s.json` — IQ3_S pack, `--vision cpu`, `--kv int8`, `--kv-resident 32768`, `--max-context 65536`, `--vram-reserve-mib 700`, `--prompt-cache-every 4096`.

Nothing was killed by hand, no OOM line in the kernel log, and the machine kept running the whole time.

Related on strata.com

Editorial links to help you install, pick models, or read release notes — not part of the upstream thread.