Issues / #3033
#3033 [Bug]: Conversation-cache metrics remain stale after refused parking
open · @oscarlius · 0 Kommentare · Auf GitHub
Server & APINVIDIA / CUDASecurityLinux
Beschreibung
## What happened After a conversation-cache park is refused under RAM pressure, `/metrics` and the Monitor card can still show the entries and bytes from the previous successful park, even though the engine has evicted those entries. Observed on Strata 0.1.45, Linux/CUDA, four RTX 4090 GPUs. This is also reproducible on current main using only `ConvCacheLog` and a temporary log file; no model or GPU is needed for the regression. ## Reproduction 1. Poll a log containing a successful park, for example: ```text strata serve: conversation cache: parked 5000 tokens in 12.0 ms; parked=38 bytes=104857600 evictions=41 snapshot_bytes=104857600 reused_kv_bytes=0 ``` 2. Append this refusal and poll again: ```text strata serve: conversation cache: skip parking (physical RAM admission; need 2128 MiB plus 32768 MiB floor, 34449 MiB available, source meminfo; evicted 38, 0 still parked) ``` 3. The parser still returns `parked=38`, the old byte count and `evictions=41`. The remaining count should be zero. The first line above is a small synthetic fixture; the refusal format and remaining-count mismatch were also reproduced from a preserved engine log. ## Cause and expected behavior `ConvCacheLog` recognizes the successful park/restore summaries and superseded-copy drops, but not the unsuccessful-parking messages. `make_room()` can evict entries before a later failure, including before the physical-RAM gate. The gate's `evicted N` is a per-attempt count, so adding it to the old cumulative total is not generally correct. The observer should use the remaining count when the log supplies it and mark values that cannot be reconstructed as unknown until an authoritative summary arrives. A refusal should not increment successful park/restore counters. Budget refusal, allocation failure and partial lines need regression coverage too. This report concerns observation only, not the cache admission policy or whether a particular workload fits in RAM. Related pressure-policy work: #2962; it is a different scope.
Mehr auf der Site
Links zu Install, Modellen, Releases.