Pull requests / #22

#22 tools: add real-time web dashboard for hardware and engine monitoring

closed · @code-martin · 0 comentarios · En GitHub

BenchmarksServer & APIMulti-GPUAMD / HIPNVIDIA / CUDAModels & quantsWindowsLinux

Descripción

## What

Adds a self-contained real-time monitoring web dashboard (`tools/dashboard_server.py`) for Strata:
- Zero external dependencies: implemented purely with Python standard library (`http.server`, `urllib`, `json`, `subprocess`, `ctypes`).
- Single-page responsive dark UI with live SVG telemetry gauges, mini trend charts, and real-time generation metrics.
- Runs alongside the Strata engine server: `python tools/dashboard_server.py --port 8081 --server http://127.0.0.1:8080`.

## Preview

![Strata Live Dashboard Preview](https://github.com/code-martin/Strata/releases/download/v0.1.6-assets/dashboard_preview.png)

## Features

- **Strata Engine Tracking**:
  - Live state, prompt/generation progress bar, current token throughput (tok/s), context window utilization, and active KV cache mode.
  - Completed request history log with prefill/gen timing, token counts, and effective speeds.
- **Hardware Telemetry (Windows & Linux)**:
  - **GPU**: Util %, VRAM usage/capacity, temperature, board power draw, active PCIe generation and lane width (e.g. `PCIe 4.0 x16` via NVML), and expert cache slots in VRAM.
  - **CPU & RAM**: Core/thread count, CPU utilization, system RAM used/total.
  - **NVMe Activity**: Per-disk read/write MB/s and load percentage (queries PDH on Windows, `/proc/diskstats` on Linux) to monitor DirectStorage/SSD streaming during MoE expert offloading.
- **Resilient**: Gracefully handles engine reconnects, idle periods, and background streaming.

## Verification

- Tested on Windows 11 with NVIDIA RTX 3080 + AMD Ryzen 9 5950X + Dual NVMe SSDs while running Strata with Qwen3.8-Flash-Next (Swift 1.5).
- Verified live polling updates, responsive gauge animations, and request history tracking.

En el sitio

Enlaces a install, modelos, releases.