Issues / #3039

#3039 [Bug]: concurrent metrics readers duplicate conversation-cache events

open · @oscarlius · 0 Kommentare · Auf GitHub

Server & APIWindows

Beschreibung

### What happened

Concurrent `GET /metrics` requests can count the same conversation-cache log event more than once. `ConvCacheLog.poll()` shares a file cursor and counters between request threads, but file reads release the GIL and the read/update sequence is not serialized.

CPU-only reproduction with one complete `conversation cache: parked ...` line:

1. Pause the first reader after its physical file read, before updating the cursor.
2. Let a second metrics request read from the same cursor.
3. Release the first reader.

One park is then reported as two. The cursor can exceed the file length, causing the next poll to rewind and replay the event again. The expected count remains one throughout.

The new direct-reader and HTTP regressions both fail on the current implementation with `(parks, restores) == (2, 0)` instead of `(1, 0)`.

### Strata version, GPU, OS

Upstream main at `477ac13`, Windows, Python 3.12.7. Reproduced using a temporary synthetic engine log and the existing `MockEngine`; no GPU or model weights are required.

### Engine log

The reproduction uses the existing `ConversationCacheCard.PARK` fixture in `serve/test_monitor.py`. No live model log is needed.

This is separate from #3033 / #3034 (refused-parking log interpretation) and #2692 (web polling versus SSE): concurrent API readers still share the same observer state.

Mehr auf der Site

Links zu Install, Modellen, Releases.