Pull requests / #3250
#3250 [Benchmark] strata_bench: read CUDA UMD version, storage and GPU power/PCIe detail
open · @JoShMiQueL · 0 コメント · GitHub で見る
BenchmarksNVIDIA / CUDAModels & quantsDocumentationWindowsLinux
本文
## Problem and result The community report came out with fields missing or wrong: - Current drivers print `CUDA UMD Version: 13.4` in `nvidia-smi`, not `CUDA Version:`, so `machine.cuda` came out null and the README said `CUDA: not read`. - Where it was read, it was the newest CUDA the **driver** supports, labelled as if the engine had been built with it (raised by autemetry-james in this thread). - Storage always said `not measured`, the GPU line carried no power limit, PCIe link, state or temperature, and Windows 11 reported as `Windows 10` because `platform.release()` is still `10`. After the change a report reads `CUDA (driver supports up to): 13.4; engine built for CUDA 13.0`, names the storage device, adds a `GPU detail (after the run, near idle)` line with power limit and draw, PCIe gen x width, pstate and temperature, and says `Windows 11`. The measurement path (HTTP suite, statistics, `results.json`) is untouched: this changes what a report prints, not what it measures. ## Related discussion Reopening of #2111, closed in the 0.2.0-beta tracker reset of 2026-10-19. The `/reopen` check passed there (area prefix in the title, template filled in, revision named), but GitHub refused the reopen itself with `state cannot be changed. The bench-tool-machine-fields branch was force-pushed or recreated.` — the branch was rebased onto `main` while the pull request was closed, so the bot's own advice applies: open a new pull request and link this one. This is that pull request; #2111 stays as the discussion thread. ## Approach - Accept both header forms from nvidia-smi and label the value as the driver's ceiling. - Read `machine.engine_build` from the engine's `BUILD.json` next to the exe, so driver ceiling and build toolkit are both in the report; no engine change. - Take the extra nvidia-smi fields from the GPUs the server uses (the selection of #1985), and label the line `(after the run, near idle)`: `collect_machine` runs a second time after the suite, so they are end-of-run, near-idle figures (qiuzan2001). - Storage: `Get-PhysicalDisk` on Windows, `system_profiler` on macOS, `lsblk -e 7` on Linux (`-e 7` drops the snap loop devices that flooded the line on a stock Ubuntu, qiuzan2001). ## Validation **Passed:** - `python tools/test_strata_bench.py` on `8f7fd37d`, rebased on `main` `e88ee39d` (0.2.0-beta): 20 tests, OK. The new checks cover the rendered report line with both CUDA numbers, the storage line, the GPU detail line, both nvidia-smi CUDA header forms, and the power/PCIe/pstate/temperature parsing. - `python tools/test_windows_amd_gpu.py` and `python tools/test_bench_host.py`, same revision: 5 and 5 tests, OK. - Live read on this PC (RTX 4060 Ti 16 GB, driver 616.92, Windows 11 build 26200): `gpu_info_local()` returned `power_limit_w=165.00`, `power_draw_w=20.25`, `pcie=gen4 x8`, `pstate=P3`, `temp_c=45`, `cuda=13.4`; `storage_info()` returned `Samsung SSD 990 EVO Plus 2TB SSD/NVMe`; the OS label resolves to `Windows 11`. - `ruff check tools/strata_bench.py tools/test_strata_bench.py`: the same 4 findings as on `main`, no new ones (3 of them are the `port` NameError below). - A report generated with this change, all fields filled, is in #2112. **Failed:** - `python tools/strata_bench.py --config strata-coder-iq1_m.json --port 9 --runs 1 --quiet` on `main` `e88ee39d` ends in `NameError: name 'port' is not defined` (`tools/strata_bench.py:1040`), from #2330. It is a separate problem and is not touched here; it blocks a default run that starts its own server on 0.2.0-beta. **Not run:** - The macOS (`system_profiler`) and Linux (`lsblk -e 7`) storage branches: no such machine here, they are read, not executed. - A full bench suite on 0.2.0-beta: blocked by the `port` NameError above. The tests drive the `--url` path against a fake server, not a real engine. ## Limits - The GPU detail line is sampled after the suite, near idle; it is not a load figure and the label says so. - `engine_build` needs a `BUILD.json` next to the exe; without one the field is left out rather than guessed. - This changes what a report prints. The measurements in #2112 were taken on engine 0.1.43 and are not re-measured here.
関連リンク
インストール・モデル・リリースへの站内リンク。