Pull requests / #2272
#2272 Monitor: the PCIe traffic from the bytes the engine moved, not from a driver counter
open · @unimo-chan · 0 comments · View on GitHub
Server & APIAMD / HIPNVIDIA / CUDAWindowsLinux
Description
## What this changes The Monitor's PCIe tile shows a traffic rate only when the driver reports one. `_Nvml.read()` fills `pcie_rx_mb` from `nvmlDeviceGetPcieThroughput`; nothing else does. An AMD card has no such counter to read: on Windows there is no NVML, the performance-counter set has no PCIe entry (only the five GPU sets and RDMA Activity) and neither `amdadlx64.dll` nor `atiadlxx.dll` exports a bus metric; on Linux amdgpu's `pcie_bw` counts messages, not bytes (#1734). The one component that does know the traffic is the engine: it staged those expert blobs itself. So the engine counts the bytes and the server turns them into a rate: - `ExpertDispatch::pcie_bytes` (`include/strata/core/expert_source.hpp`) — one blob per staged expert, in every `--pcie-mode`: the copy engine writes them into staging, kernel and direct modes read the host's blob where it lies. `offload_entries` already counted routed *entries*, not bytes. Counted in `expert_pool_dispatch_multi`, the same place `pcie_experts` is. - The engine prints the running total on `PCIE <bytes>` lines at the points where it already flushes a token line (both decode loops and both batch loops), and adds one field to `DONE`: the request's own MB. - `serve/server.py` keeps `(time, bytes)` samples and takes the slope over the same window the token rate uses (`RATE_WINDOW_S`), and reads the request's figure from the `DONE` line. It fills `gpu_pcie_rx_mb` **only when the reader says it has no traffic counter of its own** — a `has_pcie_throughput` flag, True on NVML, False on the AMD paths — so an NVIDIA card keeps showing NVML's own numbers. - The tile puts the traffic next to the bandwidth the engine measured at its own start (`PCIe probe: N GB/s host->device`, already on stderr, which the server owns as the log): `412 MB/s · 12.1% of 3.4 GB/s measured`. The request list's `+X% PCIe` badge gains the MB behind it. ## What it leaves alone - The generation half of the tile: no `pcie_gen`/`pcie_width` is added anywhere, and nothing turns a measured GB/s into a generation (Gen4 x8 and Gen3 x16 measure the same). #2137's point about the Gen headline is a separate change. - NVML's readings, and amdgpu's `pcie_bw` — #1734's question about the three sysfs integers is untouched here; this answers the rate half from the engine instead, which works on Windows too. - No new option and no default change: the engine's output gains lines, and the server only reads them. An engine older than this prints no `PCIE` lines and the tile stays as it is. ## How I checked - `python -m unittest serve.test_server.DraftCounts serve.test_server.PrometheusMetrics serve.test_server.PcieShare serve.test_server.CancelledRead serve.test_server.LiveRate serve.test_server.WebApp serve.test_server.AmdTelemetry serve.test_server.TelemetryNoDisks` on this branch (main at `a0116b2`, Python 3.12.10): **27 tests, OK** — the classes that parse the engine's `DONE` line, serve `/metrics`, and run the sampler, with the stock tests unchanged. The `DONE` parser still accepts the shorter lines of an older engine: those tests pass as they are. So the `PCIE` lines and the new `DONE` field are exercised against the mock engine, not a real one. Full `serve.test_server` on this branch: **308 tests, OK** in 101 s (Windows 11, Python 3.12.10), nothing skipped. - **Not built.** This machine has no CMake, MSVC, CUDA or SYCL toolchain, so the C++ side is not compiled here — the CUDA and SYCL trees get the same lines, and both are unverified by a compiler. A build of `src/` (CUDA/HIP) and `sycl/` before review is what AGENTS.md asks for; I could not do it, and I am saying so rather than implying it.
Related on strata.com
Editorial links to help you install, pick models, or read release notes — not part of the upstream thread.