Pull requests / #3050

#3050 serve: read conversation-cache log backlogs without dropping events

open · @oscarlius · 0 comentarios · En GitHub

Server & APIWindows

Descripción

When unread engine output exceeds 1 MiB, the cache observer jumps to the last MiB and permanently misses earlier cache events. Read from the committed cursor through a captured EOF in chunks of at most 1 MiB, advancing the cursor only through complete newline-terminated records. Preserve incomplete fragments across chunks and reread an unfinished final line on the next poll.

Add 7 local-file/HTTP regressions covering real multi-MiB backlogs, event and long-line chunk boundaries, deferred fragments, read-size bounds, concurrent appends, interrupted reads, and the existing missing-file/truncation/new-run behavior. Event parsing and native cache policy are unchanged. This addresses lost events independently of #3041's concurrent-reader serialization and #3034's refusal parser.

The observer now scans the captured unread backlog, so catch-up time scales with unread bytes. It avoids allocating the whole backlog: each read is bounded, with unfinished-line storage proportional to the longest line. It does not impose a line-length discard policy.

Validation on Windows with Python 3.12.7:

- The new suite fails on the original implementation; all 7 test methods pass after the fix.
- `python -m unittest serve.test_cache_log_backlog serve.test_monitor serve.test_server serve.test_prometheus`: 360 tests, 357 passed and 3 existing signal-related skips.
- Python compilation, `git diff --check`, and comparison against existing Ruff findings passed.
- Independent review found no actionable issues. Tests use CPU fixtures and local mock HTTP only; no native backend files changed and no live model service was restarted.

Integration with the companion completed-clear fix and #3041 merges cleanly. Adding #3034 requires resolving the adjacent `READ_MAX` comment edit: retain its refusal regexes and this PR's chunk-read comment. All 24 combined cache-observer tests pass with that resolution.

Closes #3048.

En el sitio

Enlaces a install, modelos, releases.