Issues / #596

#596 Add Conversation Cache Management UI to Monitor Tab

closed · @StoyanStAtanasov · 1 comentarios · En GitHub

Server & API

Descripción

 ## Summary
Users running multiple simultaneous agents need transparent visibility into the conversation cache behavior. Currently, KV cache slot management happens silently in the background with no way to see:
- How many agent sessions are parked in the cache
- Which prompts are reusing cached KV state
- How much memory each cached session is using
- Whether cache hits are actually occurring

## Problem
With `--conversation-cache-mib 16384 --conversation-cache-slots 8` configured, users can't:
1. Verify that agents are actually reusing the KV cache (they keep re-processing prompts)
2. See which agent is currently active vs. waiting
3. Monitor cache memory utilization over time
4. Detect stale cached sessions that should be evicted
5. Understand the cache matching strategy (prefix-based)

## Proposed Solution
Add a new **"Conversation Cache"** card to the Monitor tab that displays:

### Visual Design
- **Cache Utilization Gauge**: Show slot usage (e.g., "4/8 slots · 6.2 / 16 GB")
- **Cached Sessions Table** with columns:
  - Agent ID / Session name
  - Status badge: "Active" (green) | "Parked" (blue)
  - Cached tokens (KV pages retained)
  - Last used timestamp
  - Hit rate (if available)

## Implementation Plan

### Phase 1: Backend Data Collection (`serve/server.py`)
Extend `/metrics` to include a `cache_state` payload exposing active/parked slots, tokens retained, memory used, and last-used timestamps.

### Phase 2: Frontend Display (`serve/web/app.js` + `serve/web/index.html`)
Add a new Conversation Cache card to the Monitor tab, with a utilization gauge and a table of cached sessions.

### Phase 3: Request Tracking (Optional, v2)
Support an optional `X-Agent-ID` or `agent_id` value so different agent sessions can be labeled in the UI.

## Files to Modify
- `serve/server.py`
- `serve/web/index.html`
- `serve/web/app.js`
- `serve/web/components.css` (if needed)

## Acceptance Criteria
- [ ] Users can see how many cache slots are in use
- [ ] Users can see which sessions are active vs. parked
- [ ] Users can see memory used by the conversation cache
- [ ] Users can see whether cache reuse is happening for repeated prompts
- [ ] The Monitor tab shows this information in real time

## Related Issues
- Conversation cache feature in the engine (0.1.22+)
- Monitoring / diagnostics improvements for multi-agent usage

En el sitio

Enlaces a install, modelos, releases.