Issues / #3370
#3370 [Engine] serve: a temp cleaner that removes the idle vision encoder's work dir makes every later start fail ([WinError 267])
open · @smallmj · 2 Kommentare · Auf GitHub
Server & APINVIDIA / CUDAModels & quantsWindows
Beschreibung
### Area
[Engine] (the server and API)
### Desktop app, Terminal or source?
Source (the repo's `serve/server.py`, started by our own launcher/router)
### What happened
With `idle_unload_s` set, after the model had stayed unloaded for a while (overnight), the first request that loaded it again failed with a 503 whose message was:
```
the vision encoder failed to start: [WinError 267] The directory name is invalid. ; the next request restarts it
```
Every client retry failed the same way, seconds apart, for as long as the server process lived — only restarting the whole server healed it. The console showed a tight loop of full load attempts (the `before_load` hook ran, the free-VRAM check passed, then the vision start failed immediately; the engine was never reached):
```
[strata] model unloaded after 1800 s idle; the next request loads it again
[router] unload <other model>: 200 {"status": "not loaded"}
[strata] starting the vision encoder ...
[router] unload <other model>: 200 {"status": "not loaded"}
[strata] starting the vision encoder ...
... (x17, seconds apart, until a manual restart)
```
The engine's log had no trace of any of these attempts — no `engine started` lines (those are written before the spawn), no encoder stderr: the encoder process was never even spawned. Expected: the vision encoder starts again like it does after any other unload.
### Root cause
`Vision.work_dir()` runs once, at the server's start (`Vision.__init__`), and creates `%TEMP%\strata-vision-*`. The spawn uses `cwd=self.dir` and nothing ever checks that the directory still exists (`_start_reader()` at current HEAD; the single spawn path in earlier versions). `restart()` reuses the stale path.
While the model is unloaded, that directory sits **empty and unheld** in the temp directory — exactly what Windows' Storage Sense ("Delete temporary files that my apps aren't using", enabled on the affected machine) removes during a long idle window. The next request's `popen(cwd=<removed dir>)` raises `OSError [WinError 267]`, which `ensure_loaded` wraps as `EngineDied("the vision encoder failed to start: ...")` → 503 → the client retries → the identical failure, forever. `#914` made `shutdown()` the only remover of the dir, but nothing defends it against external removal.
Ruled out on the affected machine while diagnosing:
- **not VRAM**: with the engine resident and only ~850 MiB of VRAM free, a manually spawned `strata-vision.exe --gpu` with the config's real arguments started fine (mmproj load, warm-up, READY);
- **not a crash**: the Windows Event Log has no strata crash entries, and the encoder leaves no stderr at all — it never ran;
- the engine itself is never started during the loop.
### Smallest steps that reproduce it
1. Start the server `--lazy` with a vision config.
2. Delete the `strata-vision-*` directory in the temp directory.
3. Send any completion request (the encoder starts with the model load, not with a picture) → the 503 loop above, until the server is restarted.
### Reproduction status
Reproduced locally (twice on the affected machine, on different days; the second time with the client error text and the console loop captured).
### Strata version
engine 0.1.45 · serve/server.py from this repo (the deployed checkout was unmodified; the bug is confirmed present at current main by reading the code, e88ee39d)
### Graphics card, memory and system
RTX 3090 24 GB · 64 GB RAM · Windows 11 Pro
### Model, size and settings
Qwen3.8-Flash-Next IQ3_S · 512K context (yarn ×2, k8v4 KV) · pictures on (`--vision`, GPU encoder)
### Fix
In `Vision._start()` (the wrapper at HEAD, so the CPU-fallback path is covered too), recreate the dir when it is gone and drop the cache — the encoded-image files lived in that dir, and the cache-hit branch of `encode()` returns paths without checking they exist:
```python
if not self.dir.exists(): # a temp cleaner (Windows' Storage Sense, systemd-tmpfiles)
self.dir = self.work_dir() # can remove the idle encoder's dir while the model is
self.cache.clear() # unloaded: every start would then fail on the cwd, and the
# cache's files went with the dir
```
PR #3372 carries the fix and a regression test (`serve/test_vision_dir.py`).
Mehr auf der Site
Links zu Install, Modellen, Releases.