Issues / #2496

#2496 [Bug]: serve drops the connection when the engine program cannot be started

open · @devYRPauli · 1 コメント · GitHub で見る

Server & APIWindows

本文

### What happened

When the server cannot start the engine program, the request that starts the engine gets no HTTP answer. The client sees a closed connection with no status line. This happens on a lazy first load and on a restart after the engine stopped.

Any `OSError` from `subprocess.Popen` takes this path. Three examples:

- The engine program is missing (`FileNotFoundError`).
- The file is not executable (`PermissionError`).
- Windows blocks the program (#735, WinError 4551 or 1260). I simulated this case at the `subprocess.Popen` boundary.

To reproduce with no GPU, save this as `repro.py` in the repo root and run `python repro.py`:

```python
import json
import urllib.request
from pathlib import Path
from serve.server import ByteTokenizer, ChatTemplate, Service, StrataEngine, serve

eng = StrataEngine("/no/such/strata", [], log="strata-test.log", lazy=True)
svc = Service(eng, ByteTokenizer(), ChatTemplate(Path("serve/chat_template.jinja")))
httpd = serve(svc, port=0)
url = f"http://127.0.0.1:{httpd.server_address[1]}/v1/chat/completions"
body = json.dumps({"messages": [{"role": "user", "content": "hi"}], "max_tokens": 8}).encode()
try:
    urllib.request.urlopen(urllib.request.Request(url, body, {"Content-Type": "application/json"}))
except Exception as e:
    print(type(e).__name__, getattr(e, "code", ""), e.read().decode() if hasattr(e, "read") else e)
```

On main 296b59d2 the script prints this line, and the server prints a `FileNotFoundError` traceback:

```
RemoteDisconnected  Remote end closed connection without response
```

Expected: a 503 with the reason. An engine that exits before READY already gets one (#1527). The comment above `raise EngineDied` in `StrataEngine.__init__` asks for the same result.

I measured each of the three cases on main 296b59d2 with 6 requests: chat, messages and responses, streamed and not.

- All 18 requests end with a closed connection.
- `/health` still reports `status: ok`.
- The API monitor records each request as `disconnected`.
- Each failed start leaves one start narrator thread running. The last failed start also keeps its engine log open.
- The narrator prints "still starting (N s) - please wait" every 20 s and never stops. After 21 s the server window held 18 such lines.
- A later request recovers when the engine program exists again.

Cause: `StrataEngine.__init__` calls `popen()` with no handler. The `OSError` passes through `restart()`, which catches only `RuntimeError`, and through `Service.ensure_loaded()`. The request handler's `except OSError` branch records `disconnected` and re-raises, so `socketserver` closes the socket. On this path `__init__` also never sets the `loading` event and never closes `self.log`.

I will open a PR with a fix and a regression test.

### Strata version, GPU, OS

main 296b59d2 (serve/ only, no engine build), no GPU, macOS 27 (Darwin 27.0.0), Python 3.12.14

### Engine log

_No response_

関連リンク

インストール・モデル・リリースへの站内リンク。