Pull requests / #3091

#3091 monitor: show every parallel request + fixed-height slot dots (#465)

open · @superlee-terry · 0 commentaires · Sur GitHub

BenchmarksServer & APIMulti-GPUNVIDIA / CUDA

Description

## What

Two web-monitor commits (previously part of #2738, now split out; all engine changes from that PR are dropped — see the closing comment there for the 2x3090 A/B data).

**1. Show every parallel request (#465).** With `--batch N`, the Model state card's single progress bar only ever showed the newest request, so concurrent streams were invisible. The card now renders one row per batch slot: reading progress, decoding (tokens, tok/s, elapsed), or idle with the tokens it holds for the next turn — plus solo-path / waiting-for-slot / queued counts.

**2. Slot dots on the head row.** The extra-counts line under the grid appeared and disappeared with load, so the card height jumped between single and parallel requests. It is replaced by one dot per slot on the state-card head row, right-aligned, text-free: grey = idle, blue = reading, green = generating, yellow = queued/waits, red = error; hovering a dot shows what that slot is doing. The card height is now constant.

Dot and row counts come from `/metrics`' `live.slots` (the server reports one entry per `--batch` slot), so they follow `parallel`/`--batch` dynamically — 2 slots render 2 dots, 4 render 4. `--spec` is the MTP budget and is unrelated to slots.

## Test

Live on 2x RTX 3090, batch 3: 3 concurrent streams render 3 generating rows + 3 green dots; idle returns to grey; state card height measured constant (197px) across idle/parallel transitions. `serve/web` static files only — no server, engine, or protocol changes.

Sur le site

Liens install, modèles, releases.