Pull requests / #555

#555 serve: a request's combined image file is deleted when the request is refused or never runs

closed · @sergqwer · 0 comentarios · En GitHub

Server & APIModels & quants

Descripción

`prepare()` writes a request's pictures into one `req-<id>.sve` in the vision directory (~10 MB a picture), and only `run()`'s `finally` deletes it. A request that never reaches `run()` leaves the file there for good. That covers:
- a request refused after the write: `EngineStarting` (503), "no room to answer", "exceeds the context";
- a client that disconnects between `prepare()` and `run()`.

Clients retry 503s while the engine restarts, so the temp directory grows by a file per retry.

**Fix** (`serve/server.py`):
- The combined file is written only after the checks that can refuse the request.
- Its path is recorded before the write, so a half-written file is found too.
- `run()` takes the path over as it starts.
- A new `Service.drop_embeddings()`, called in the handler's `finally`, deletes a file no `run()` took.

Tests (`VisionTempFiles`): refusals for EngineStarting, no room and exceeds leave no `.sve`. So do, over HTTP, a 503, a 400, a client gone after `prepare()`, and normal requests. Without the fix 3 and 6 files were left. Serve tests: 174 OK (7 skipped).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01VZy1yKaDDiA8a7svdwaHio

En el sitio

Enlaces a install, modelos, releases.