#2341 /metrics/prometheus gauges never decay — stale non-zero values served while server is fully idle

本文

* Engine: Strata 0.1.43.1 (endpoint :8080, /metrics/prometheus, Prometheus text format)
                                                                                                                                           
# Description        
                                                                                                                                           
We scrape Strata's Prometheus-format endpoint with Prometheus (15 s interval) and graph it in Grafana. While the server is completely idle, the exposition output is byte-for-byte frozen at the values from the last handled request. 

Verified by scraping twice 45 seconds apart with zero traffic: identical output, including:
                                                                                                                                      
* llamacpp:prompt_tokens_seconds 1408.1173761205446                                                                                    
* llamacpp:predicted_tokens_seconds 58.70608143248928

…served while the server had handled no request for many minutes (requests_processing 0, requests_deferred 0).                         
                                                                                                                                           
# What's fine        

* The *_total counters staying pinned is correct Prometheus counter semantics (cumulative since start) — no change wanted there.
* rate() over them correctly yields 0 when idle.                                                                                         
* requests_processing / requests_deferred correctly report 0 when idle.
                                                                                                                                           
# The problem                                                                                                                            
                                                                                                                                           
* llamacpp:prompt_tokens_seconds and llamacpp:predicted_tokens_seconds are exposed as plain gauges but hold lifetime-average throughput (tok/s). Because they never decay, the endpoint advertises e.g. "58 tok/s decode speed" indefinitely after the last request, with no timestamp or freshness signal in the text format to indicate otherwise. Any operator or alerting rule reading the raw endpoint — or any dashboard panel that plots the gauge directly instead of a rate() ratio — gets a confidently wrong "server is fast and was recently busy" signal.
            
# Expected behavior (any of these would fix it)

1. Decay the throughput gauges to 0 when no request has completed in the last scrape interval (or N seconds), matching what an    operator means by "current speed"; or
2. Only expose the gauges while a request is in flight / recently completed (omit the sample when stale — Prometheus treats absent series as no-data, which dashboards render as a gap); or
3. Minimum: rename/document them as lifetime averages (e.g. ..._average_tokens_per_second_lifetime) so consumers can't mistake them for instantaneous throughput.                                                                                                          
                                                                                                                                           
# Environment notes                                                                                                                      
                       
* The metrics endpoint is served by BaseHTTP/0.6 Python/3.14.4 (HTTP/1.0 responses) — fine for Prometheus, just noting it's a separate path from the inference server.
* llama.cpp's native /metrics exposes the same two metric names with the same lifetime-average semantics, so we understand this is partly inherited behavior — but llama.cpp also exposes per-slot KV utilization and spec_decode_* counters that give dashboards honest activity signals; Strata currently exposes neither, so these two stale gauges are the only "speed" telemetry available and their staleness is much more consequential.                                                                                                  
                                                                                                                                           
# Repro                                                                                                                                  

1. Start Strata, send one chat completion.
2. Wait 5+ minutes with no traffic.                                                                                                    
3. curl -H "Authorization: Bearer <key>" http://<host>:8080/metrics/prometheus
4. Observe llamacpp:predicted_tokens_seconds still non-zero; repeat after another 5 min — identical value.