Issues / #2257

#2257 SIGTERM during a running request restarts the engine instead of exiting (two 47 GB loads, OOM kill)

closed · @SeriousJul · 0 comments · View on GitHub

Server & APINVIDIA / CUDAModels & quantsLinux

Description

## What happens

`serve/server.py` run under a supervisor that stops the model by signalling the
whole process group does not exit. The engine children receive SIGTERM with the
group, the request that was running sees its engine die, and the request path
starts a new engine. The server prints `stopped`, its port closes, and the
engine it just restarted keeps loading 46.84 GiB. The supervisor, seeing the
port gone, starts a second instance. Two 47 GB loads at once on a 60 GiB box:
`systemd-oomd` kills the unit.

## Environment

- engine 0.1.40 (`engine/BUILD.json`, `src 6f298dff754d187f`), Linux, CUDA
- RTX 4090, 60 GiB RAM, 120 GiB swap
- config `strata-iq3_s.json`: 125B MoE pack, `--expert-cache auto`, `--vision`,
  `--max-context 262144`, `--kv int8`
- supervisor: llama-swap, which starts `cmd` with `Setpgid` and signals the group

## Steps

1. `python serve/server.py --engine strata --config strata-iq3_s.json --port 6307`
2. Send a request that streams for a while (`max_tokens: 4000`).
3. While it is still streaming, send SIGTERM to the server process and to its
   engine children, the way a process-group stop does.

## Actual

```
[strata] stopping (Ctrl+C again to end the engine at once) ...
[strata] the engine stopped unexpectedly (exit code -15). ... The next request starts the engine again.
[strata] starting the vision encoder again ...
[strata] the engine had stopped (exit code None); starting it again (a minute or two) ...
[strata] stopped
loading the vision encoder ...
loading the model (the first start takes a minute or two) ...
llama-swap.service: systemd-oomd killed 75 process(es) in this unit.
llama-swap.service: Failed with result 'oom-kill'.
```

The restart comes from `Service.ensure_loaded()` (`serve/server.py:2356` and
`serve/server.py:2364`), which runs after the shutdown has begun. The process
says `stopped` and stops answering, but it is loading the model again, so RAM
keeps falling while the supervisor starts a fresh instance.

Reproduced twice in a row with the steps above, and seen twice more in the
llama-swap journal on 2026-10-13 at 11:44 and 11:52. A SIGTERM while the server
is idle shuts down cleanly in about 2 s, so the request path is what races it.

## Expected

Once SIGTERM or Ctrl+C has been received, no request path starts the engine or
the vision encoder again. The process ends and frees the pinned arena.

## Suggested fix

Set a "shutting down" flag in the SIGTERM / KeyboardInterrupt path, before
`engine.close()` sends QUIT, and have `Service.ensure_loaded()`,
`StrataEngine.restart()` and `Vision.restart()` refuse to start anything once it
is set. The in-flight request then ends with the existing `EngineDied` error
instead of reloading the model. The server's own shutdown is what kills the
engine, so the restart is triggered by the shutdown itself.

## Notes

- Adjacent to #1872 (SIGTERM and the clean shutdown path), but that one is about
  `--open`'s browser launch; this is the request path restarting the engine.
- The supervisor can work around it by killing the engine again after the stop,
  which is what my test script does. That is not a fix, and it is the reason the
  OOM is easy to miss: the second load looks like the next model starting.

Related on strata.com

Editorial links to help you install, pick models, or read release notes — not part of the upstream thread.