Pull requests / #2661

#2661 serve: the Profile page (/profile, opt-in "profile": true): the last request's breakdown as it happens, and a run of tools/strata_profile.py from the page

open · @qiuzan2001 · 0 comments · View on GitHub

BenchmarksSetup & installServer & APIAMD / HIPNVIDIA / CUDAModels & quantsSecurityDocumentationWindowsLinux

Description

## What it adds

An opt-in **Profile page** at `/profile` (`"profile": true` in `strata-<model>.json`, or `serve/server.py --profile`; also a switch in the About tab's Model settings, used from the next start):

- the engine starts with `STRATA_PREFILL_TIMING=1` and `STRATA_DECODE_TIMING=1` in its environment (a value the user set for either stays), so every request writes its breakdown to the engine log;
- the page shows the **last finished request** as it happens: the prompt read's GPU time by group (experts streamed over PCIe, dequant, expert GEMM, attention, GDN, other) and the decode window's milliseconds by what the host waited for (the card, the experts computed on the CPU, the draft, the adaptive tier), with the hit rate, the PCIe share and this request's KV streaming reads. `GET /profile/last` reads the lines from the tail of the engine log with the parsers of `tools/strata_profile.py` (#2660, which this branch includes: its first commit is that PR);
- a **Tune this PC** button: `POST /profile/tune` (own page only) unloads the model (409 while a request runs, as `/unload`) and runs setup's calibration in a child process (`python -m serve.profile_page --calibrate <config>` calls `setup.calibrate_config`: tools/calibrate.py's measurements, the winners written into `strata-<model>.json` and remembered in the settings file, exactly as `setup.py --calibrate` does). When it ends, the calibrated flags and the config's `env` go into the engine's start command (`apply_to_spawn`), so the model loads with them on the next request. While it runs the server does not load its engine beside the calibration's (a request gets 503 with `Retry-After`, as when the GPU is in use by another program), `/profile/run` and the Settings view's `POST /config` answer 409, and the page asks before starting (setup's own estimate: roughly 15-30 minutes, longer on a slow card). As setup, the page offers it on NVIDIA only and not for replicas (the status says why);
- a **Run the profile** button: `POST /profile/run` (Strata's own page only, as `/settings`: JSON, no foreign Origin, the API key when one is set) starts `tools/strata_profile.py` against this server, `GET /profile/status` follows it and returns the written report (`bench/results/<date>-profile-<gpu>-<model>/`), shown as the same tables and findings. The API key goes to the tool through its environment, not the command line.

Off (the default) nothing is set in the engine's environment and the five paths answer 404, so the default path is unchanged. The page is built like `/api-monitor` (same tokens.css, no CDN, server data only as text).

## Measured (RTX 5070 Ti 16 GB, Ryzen 7 5800X3D, Linux, Strata 0.1.44.1, Swift 1.5 IQ3_S at 262K, INT8 KV, `--kv-resident 32768`)

The cost of leaving the two switches on, `tools/ab_engine.py`, 3 alternating rounds with a restart per arm, greedy, 256 new tokens:

| Request | off | on | per pair |
| --- | ---: | ---: | --- |
| 8K-token prompt read (6,000 words), prompt tok/s | 2,699-2,710 | 2,582-2,592 | -4.7 / -4.3 / -3.9% (3 of 3 slower) |
| the same request's decode, tok/s | 70.4-75.6 | 71.0-75.2 | +6.8 / -1.3 / -6.1% |
| three short questions, decode tok/s (medians) | 78.0 / 79.8 / 77.1 | 78.3 / 75.8 / 77.4 | -7% to +2%, no consistent sign |

So the prompt path pays about 4% for its event marks on this card; the decode's timestamps are within the noise. The raw rows are in the PR (`bench/results/2026-10-15-timing-switches-rtx-5070-ti/`); docs/DETAILS.md says so beside the page and tells the user to turn it on while looking, off when done.

Real check on the same box: the server started with `--profile` prints the switches in its start line; `/profile/last` answered `request: null` before any request and the breakdown of a 64-token answer after it; a run started from the page (`POST /profile/run`) and `GET /profile/status` returned its lines and the report; a POST with a foreign `Origin` got 403. The run from the page (`?long=8192`, the model already loaded) took 42 s and its findings matched the command-line run's (2K read 43% + 15% waiting for expert copies; decode windows 9.5-10.2 ms of card and 11-18 ms of experts computed on the CPU). A 20-token prompt has no prefill timing line (the page says so and shows the decode window). The Tune button, on the same box from a copy of the config: a second POST while it ran got 409, `/health` said unloaded, the calibration ended after 681 s with `--pcie-frac 0.35, --spec-min-p 0.70, --adapt-every 1, --adapt-swaps 160, --adapt-decay 0.97 (96.6 tok/s)` written into the config, the server printed its "tuned for this PC" line, and the next request loaded the engine with those five flags on its command line (checked with `ps`). This check ran before the second review's guards were added; those are covered by the unit tests.

## Checked

- `python3 -m unittest serve.test_profile` 20/20; `serve.test_runconfig`, `serve.test_security`, `serve.test_monitor`, `serve.test_lifecycle` unchanged (71 together); `tools/test_strata_profile.py` 28/28. `node --check serve/web/profile.js`.
- Default path: without the key, `child_env` is unchanged and the paths answer 404 (tested).

## Not done

- The page was checked with curl and the real engine on Linux; not opened in every browser (the script uses the same APIs as monitor.js).
- Replica servers (several engines): the switches reach every replica through `child_env`, but `/profile/last` reads the primary engine's log only; not run.
- A reviewer (a sub-agent) read the diff before this was opened; its findings (the module name shadowing the stdlib's `profile`, the report path read back only from the results folder, a pump race, the server's own host for the tool, the run's end at shutdown, the unmeasured "three minutes", the A/B data not in the repo) are in this revision; a second pass on the tuning button added the no-load guard while it runs, the replica and HIP refusals, the result applied before the run is published as done, the calibration's own variables only, and the wording.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Related on this site

Quick links to related pages.