Pull requests / #2204

#2204 Monitor: token usage and cost per day, priced, with the folder's models merged or shown one by one

open · @pipti · 0 commentaires · Sur GitHub

Server & APIAMD / HIPNVIDIA / CUDADocumentation

Description

Issue: Resolves #900

## Summary

The Monitor gains a **Token usage and cost** card. It keeps, per day, what each request used (prompt, reused, written tokens) and what the machine actually drew for it (watt-hours, folded in from the Power card's own sampler), so the figures survive a restart and describe real days rather than one sitting. Everything is priced at your own rates, set in About > Token pricing.

A run folder normally holds the usage file of every model one has run. The card adds them all up by default; the one choice **Merged / By model** — on the card's own title bar and in About > Settings — shows them one by one instead, both as a line per model and as one row per model and day in the per-day list.

Three commits, one feature. Nothing in the engine, the request path or any default changes.

## What changed

- **serve/usage.py** (new, 294 lines) — `UsageStore` keeps the per-day counts in `strata-<model>.usage.json` beside the run config (tokens, watt-hours, seconds sampled) and prices them: `DEFAULT_PRICING` is CNY 2 / 0.4 / 8 per million tokens (input / cached input / output), 0.55 per kWh, 100 W for the rest of the box — all settable, kept in `strata-<model>.pricing.json`. `read_days()` is the one reader of a usage file, `days_view()` the one builder of the card's rows (today, the last 14 days newest first, the all-time sum, each priced), and `summed()` adds several models' days up. `ModelsUsage` names each model after the `model_name` of the run config beside its file (the file's stem when there is none; two configs of one model are one model), and reads the other files at most once per poll, keyed by mtime and size. A file that is missing or half written reads as the days it holds.
- **serve/server.py** — `Service.usage_view()` is what GET /usage, POST /pricing and the usage block of GET /metrics answer: the folder merged, plus `models` for the per-model view, every model priced with this server's prices. The running model's counts come from the live store rather than its file, so the seconds the sampler holds in memory are not lost; the other models come from disk, which their own server saves every `SAVE_EVERY_S`. `start_telemetry()` points the sampler's new `on_sample(sample, dt)` at `Service._sample_power`, which folds `gpu_power` into the store and splits the busy share at `GPU_BUSY_PCT` (the same line `gpu_busy()` draws). A server started without a run config keeps the numbers in memory only, as before.
- **serve/telemetry.py** — the power sampler gained `on_sample(sample, dt)`, called with the time the reading actually covers; a hook that raises can never end the sampler thread.
- **serve/web/** — the Monitor card (tokens, electricity and total; a line per model with its today and all-time figures when the per-model view is on), its per-day list (a kWh column; a Model column and a row per model and day in the per-model view), the Token pricing card in About, and the one choice in its two places, kept in this browser. `app.css` grew `.seg--2` (two choices) and `.seg--sm` (the card's small one). A server that sends no per-model view keeps the merged card, as before.
- **serve/test_usage.py** (new, 368 lines) — 18 tests: the store's days and prices, a hand-edited file with nonsense in it, the folder's names, sums and whose counts win, POST /pricing, and the /usage + /metrics payload in both views against the mock engine.
- **docs/DETAILS.md** — what the card keeps, where the two files live, and where the choice is.

## Extra Notes

- **Overlap:** #1876 and #1925 put GPU energy and its price on the Power card. This bills the whole box on the usage card instead — the card's measured watt-hours plus `other_watts` over the seconds sampled. Happy to drop that commit or fold it into theirs if you would rather keep the two apart; the token card stands on its own without it.
- **Verified:** the full suite on this branch rebased onto a0116b27 (`python -m unittest discover -s serve -t . -p "test_*.py"`) runs 743 tests, 12 skipped, and reports one error — `serve.test_server.HipEnvGuard.test_only_for_hip`, which does `Path(r"Z:\definitely\not\here").exists()` in `hip_env_guard` and hits a disconnected network drive here (`OSError: [WinError 1331]`). A clean a0116b27 checkout on this same machine fails the same test the same way (`Ran 4 tests ... FAILED (errors=1)`), so it is this PC's `Z:` drive, not the change; `serve.test_usage` is 18 tests, all passing. On the machine here (RTX 5090 32 GB, Ultra 9 285K, 95 GB) a folder of three models came out as 1,937 requests / 157,142,730 prompt / 10.53 kWh / ¥142.25 merged, and 857 + 469 + 611 requests with ¥68.83 + ¥40.53 + ¥32.89 split per model, which is what the per-day files hold. Both views and both switches were clicked through in a headless browser against a mock engine, with no console errors.
- **Leaves alone:** no new dependencies, no change to the engine, the flags, the request path or any default; the two JSON files are plain and can be read, edited or deleted by hand, and a server without a run config keeps its counts in memory only. The card hides itself when the payload carries no usage block, so an unmodified older server shows exactly the page it did before — if you would rather the card be opt-in on this build too (hidden until a flag or a file turns it on), that is a small change and I will make it.

Sur le site

Liens install, modèles, releases.