改动 / #2858

#2858 serve: the Monitor's Workers card - one widget per batch slot, each with its own tok/s (#465)

open · @street-tripy · 0 评论 · 去 GitHub 看

BenchmarksSetup & installServer & APIAMD / HIPNVIDIA / CUDAModels & quants

说明

## Title
Issue: Resolves #2857

## Summary
With `"parallel": N` the server runs several requests at once, but the Monitor still shows one
request's numbers: the header pill reads the newest request's rate and the Model state card
shows one request's progress. There is no way to see on the page whether the requests are
decoding together or one after the other.

This adds a **Workers** card to the Monitor — one widget per batch slot with its own state, live
tok/s, tokens written and context fill — and makes the header pill name every decoding stream
(`2 decoding · 80.4 + 76.1 tok/s`). It is opt-in by construction: the card is in the markup
hidden and is shown only when `/metrics` reports `live.parallel`, which the server sets only for
a run config with `"parallel": N`.

## What changed
- `serve/web/index.html` (+7): the `workers-card` markup after *Model state*, `hidden` by
  default, with `workers-sum`, `workers` and `workers-note`.
- `serve/web/app.js` (+63 / -1): `workerWidgetsFor()` builds one widget per slot the first time
  and reuses it, so state and progress transitions animate instead of being replaced every
  second; `renderWorkers()` fills it from the existing `live.slots` and is called from
  `renderMonitor()`. The pill line gained a branch: when two or more slots are decoding it lists
  their rates; otherwise it is the old text.
- `serve/web/app.css` (+17): the card's styles, from existing design tokens only (no new
  colors, fonts or selectors outside `.worker*`).
- `serve/test_monitor.py` (+11): `test_the_workers_card_is_on_the_page_and_opt_in` — the same
  page assertion as `test_the_card_is_on_the_page`, plus that the card ships `hidden` and is
  gated on `live.parallel`.

## Extra Notes
- **Nothing outside `serve/web` and this test.** No server, setup or engine file changed, so
  there is nothing to build: no CUDA, HIP or SYCL file is touched.
- **The default path is unchanged.** Checked on this branch with a `/metrics` payload without
  `parallel`: the card stays hidden, no widget is built, and the pill reads
  `Generating · 79.8 tok/s` as before. With `parallel: 2` both slots render with their own rate.
  No console errors in either state.
- **Tests:** `python -m unittest serve.test_monitor serve.test_runconfig` → 20 tests, all pass
  (19 on `main` + the new one). `node --check serve/web/app.js` is clean.
- **No new API.** The widget reads `live.slots` as `slots_view()` in `serve/server.py` already
  publishes it; the server and its tests are untouched.
- The widgets are filled one second at a time from the existing `poll()`, the way the request
  table already is; each widget keeps its own context bar against `engine.max_context`.
- Left alone on purpose: the standalone API Monitor page (`serve/web/monitor.html`), the Chat
  and About tabs, and every hardware card.
- Possibly touching the same markup as #1114 (rearrangeable Monitor layout) and #2317 (one line
  per card on a layer split); no functional overlap — happy to rebase if either lands first.

本站相关内容

相关页面的快捷入口。