Pull requests / #2023

#2023 Video input: bounded preparation progress and cancellation follow-up

open · @constantindjonkam · 0 comments · View on GitHub

Server & APIMulti-GPUAMD / HIPNVIDIA / CUDAModels & quantsSecurity

Description

## Summary

Follow-up to #1609, based on v0.1.43. The six original #1609 commits are preserved as individual cherry-picks with their original authorship; this branch adds the serving/lifecycle work needed for long video preparation.

- report bounded `VPROG <frames_done> <frames_total>` updates from `strata-vision` once per 64-frame batch;
- poll encoder replies instead of blocking for the whole video, with validation and the existing timeout;
- bound concurrent video preparation by the engine batch capacity and expose queue/preparation/progress state in `/status` and `/metrics`;
- cancel queued and in-progress preparation when a client disconnects, kill/wait for the encoder, remove partial files, and restart the encoder on the next request;
- propagate cancellation through Chat Completions, Responses, Anthropic, and MCP preparation paths;
- show preparation stage and bounded progress in the web UI.

The follow-up does not combine the incompatible SVE2 video implementation from #1327.

## Validation

- clean isolated CUDA engine and vision builds under `/tmp/strata-video-followup-build`;
- video/progress/cancellation tests, `ClientHangUp`, and the isolated Responses sampler test pass;
- `gguf_split_test` and `gguf_reader_test` pass;
- the combined `serve.test_server serve.test_frontend serve.test_responses serve.test_parallel` run covered 388 tests with one skip. The existing order-sensitive `NoLeakedSampler.test_server_close_ends_the_sampler` assertion passed in one run and failed in a later full-order rerun, but passes alone; it is unrelated to this video change;
- live AP-Q5 CUDA smoke test on RTX 3090 + RTX 5080: OpenAI Chat Completions video returned HTTP 200, Responses `input_video` returned HTTP 200, `/metrics` observed `preparing_video` and `VPROG 64/122`, client-disconnect cancellation returned to idle, and the next video request succeeded after encoder restart. GPU 2's unrelated workload was not disturbed.

Related on strata.com

Editorial links to help you install, pick models, or read release notes — not part of the upstream thread.