#4264 [Engine] Vision: keep image ownership through request preparation
open · @Tsury · 0 commentaires · Sur GitHub
Server & APIAMD / HIPModels & quantsLinux
Description
## Problem and result The image cache keeps its records during `encode_all`, but the service releases the FIFO before it prepares the combined embedding file. Another request can evict those files between encoding and the stat/copy step, so the first request fails or copies a missing file. A refused request with more than 64 cached images can also leave the cache above its bound. After this change, a request owns its images from encoding through the combined-file copy. Eviction waits for the outermost batch, and the cache trims back to 64 afterward. Encoder pipes close on teardown, and failed startup cleans up. A native decoder refusal retries once through the existing Pillow normalization path; conversion failures still remove their temporary files. This replaces #2286 (closed in the 0.2.0-beta tracker reset), rebased on current `main`. ## Related discussion #2286 (this change on the old base, with the review note about `Vision.__new__` tests). #1072 and #1207 fixed retention during encoding; this covers the remaining interval through the combined-file copy. ## Approach A reentrant batch counter (`cache_batches`) defers eviction until the outermost batch exits, so nested `encode_all`/prepare calls keep ownership without a second lock. The FIFO stays held through validation and copying. The existing `VisionCacheEviction` tests that build `Vision` with `__new__` now set `cache_batches` (the review note on #2286). No config, default or API change. The longer FIFO hold protects ownership; its throughput cost is unmeasured. ## Validation **Passed:** - `python -m unittest serve.test_vision_cache serve.test_server.VisionDecode serve.test_server.ImageMarkers serve.test_server.VisionTempFiles serve.test_server.VisionShutdown serve.test_server.VisionCacheEviction serve.test_vision_elastic` on this branch (main `4b9e05a2` + this commit), Linux, Python 3.14, CPU only: 30 tests OK. On stock `4b9e05a2` with the same tests: 16 failures and 2 errors (eviction before stat, eviction between stat and copy, out-of-order records for >64 images, untrimmed failure paths, the decoder retry). - `python -m unittest discover -s serve -p 'test_*.py'` on this branch: 1046 tests, 1 failure (below), 12 skipped. **Failed:** - `serve.test_keepawake.KeepAwakeTests.test_service_holds_for_the_whole_request_including_early_close` fails intermittently on this branch and on stock `4b9e05a2` alike (2 of 3 runs on stock here). It is a timing test unrelated to vision. **Not run:** - No GPU, model or device build. The race tests schedule two requests against the real cache and copy boundary without an encoder; they do not qualify a new image decoder. ## Limits The FIFO now holds longer per image request; I did not measure the throughput cost. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Sur le site
Liens install, modèles, releases.