Pull requests / #2858
#2858 serve: the Monitor's Workers card - one widget per batch slot, each with its own tok/s (#465)
open · @street-tripy · 0 commentaires · Sur GitHub
BenchmarksSetup & installServer & APIAMD / HIPNVIDIA / CUDAModels & quants
Description
## Title Issue: Resolves #2857 ## Summary With `"parallel": N` the server runs several requests at once, but the Monitor still shows one request's numbers: the header pill reads the newest request's rate and the Model state card shows one request's progress. There is no way to see on the page whether the requests are decoding together or one after the other. This adds a **Workers** card to the Monitor — one widget per batch slot with its own state, live tok/s, tokens written and context fill — and makes the header pill name every decoding stream (`2 decoding · 80.4 + 76.1 tok/s`). It is opt-in by construction: the card is in the markup hidden and is shown only when `/metrics` reports `live.parallel`, which the server sets only for a run config with `"parallel": N`. ## What changed - `serve/web/index.html` (+7): the `workers-card` markup after *Model state*, `hidden` by default, with `workers-sum`, `workers` and `workers-note`. - `serve/web/app.js` (+63 / -1): `workerWidgetsFor()` builds one widget per slot the first time and reuses it, so state and progress transitions animate instead of being replaced every second; `renderWorkers()` fills it from the existing `live.slots` and is called from `renderMonitor()`. The pill line gained a branch: when two or more slots are decoding it lists their rates; otherwise it is the old text. - `serve/web/app.css` (+17): the card's styles, from existing design tokens only (no new colors, fonts or selectors outside `.worker*`). - `serve/test_monitor.py` (+11): `test_the_workers_card_is_on_the_page_and_opt_in` — the same page assertion as `test_the_card_is_on_the_page`, plus that the card ships `hidden` and is gated on `live.parallel`. ## Extra Notes - **Nothing outside `serve/web` and this test.** No server, setup or engine file changed, so there is nothing to build: no CUDA, HIP or SYCL file is touched. - **The default path is unchanged.** Checked on this branch with a `/metrics` payload without `parallel`: the card stays hidden, no widget is built, and the pill reads `Generating · 79.8 tok/s` as before. With `parallel: 2` both slots render with their own rate. No console errors in either state. - **Tests:** `python -m unittest serve.test_monitor serve.test_runconfig` → 20 tests, all pass (19 on `main` + the new one). `node --check serve/web/app.js` is clean. - **No new API.** The widget reads `live.slots` as `slots_view()` in `serve/server.py` already publishes it; the server and its tests are untouched. - The widgets are filled one second at a time from the existing `poll()`, the way the request table already is; each widget keeps its own context bar against `engine.max_context`. - Left alone on purpose: the standalone API Monitor page (`serve/web/monitor.html`), the Chat and About tabs, and every hardware card. - Possibly touching the same markup as #1114 (rearrangeable Monitor layout) and #2317 (one line per card on a layer split); no functional overlap — happy to rebase if either lands first.
Sur le site
Liens install, modèles, releases.