Issues / #2496
#2496 [Bug]: serve drops the connection when the engine program cannot be started
open · @devYRPauli · 1 コメント · GitHub で見る
本文
### What happened
When the server cannot start the engine program, the request that starts the engine gets no HTTP answer. The client sees a closed connection with no status line. This happens on a lazy first load and on a restart after the engine stopped.
Any `OSError` from `subprocess.Popen` takes this path. Three examples:
- The engine program is missing (`FileNotFoundError`).
- The file is not executable (`PermissionError`).
- Windows blocks the program (#735, WinError 4551 or 1260). I simulated this case at the `subprocess.Popen` boundary.
To reproduce with no GPU, save this as `repro.py` in the repo root and run `python repro.py`:
```python
import json
import urllib.request
from pathlib import Path
from serve.server import ByteTokenizer, ChatTemplate, Service, StrataEngine, serve
eng = StrataEngine("/no/such/strata", [], log="strata-test.log", lazy=True)
svc = Service(eng, ByteTokenizer(), ChatTemplate(Path("serve/chat_template.jinja")))
httpd = serve(svc, port=0)
url = f"http://127.0.0.1:{httpd.server_address[1]}/v1/chat/completions"
body = json.dumps({"messages": [{"role": "user", "content": "hi"}], "max_tokens": 8}).encode()
try:
urllib.request.urlopen(urllib.request.Request(url, body, {"Content-Type": "application/json"}))
except Exception as e:
print(type(e).__name__, getattr(e, "code", ""), e.read().decode() if hasattr(e, "read") else e)
```
On main 296b59d2 the script prints this line, and the server prints a `FileNotFoundError` traceback:
```
RemoteDisconnected Remote end closed connection without response
```
Expected: a 503 with the reason. An engine that exits before READY already gets one (#1527). The comment above `raise EngineDied` in `StrataEngine.__init__` asks for the same result.
I measured each of the three cases on main 296b59d2 with 6 requests: chat, messages and responses, streamed and not.
- All 18 requests end with a closed connection.
- `/health` still reports `status: ok`.
- The API monitor records each request as `disconnected`.
- Each failed start leaves one start narrator thread running. The last failed start also keeps its engine log open.
- The narrator prints "still starting (N s) - please wait" every 20 s and never stops. After 21 s the server window held 18 such lines.
- A later request recovers when the engine program exists again.
Cause: `StrataEngine.__init__` calls `popen()` with no handler. The `OSError` passes through `restart()`, which catches only `RuntimeError`, and through `Service.ensure_loaded()`. The request handler's `except OSError` branch records `disconnected` and re-raises, so `socketserver` closes the socket. On this path `__init__` also never sets the `loading` event and never closes `self.log`.
I will open a PR with a fix and a regression test.
### Strata version, GPU, OS
main 296b59d2 (serve/ only, no engine build), no GPU, macOS 27 (Darwin 27.0.0), Python 3.12.14
### Engine log
_No response_
関連リンク
インストール・モデル・リリースへの站内リンク。