反馈 / #2062

#2062 strata_bench.py mislabels mixed-GPU hosts and single-GPU runs on multi-GPU hosts

open · @MinhHaDuong · 2 评论 · 去 GitHub 看

BenchmarksSetup & installMulti-GPUNVIDIA / CUDAModels & quantsLinux

说明

`tools/strata_bench.py` (v0.1.43) labels the hardware wrong on a host with two different GPUs. The measurements themselves are fine.

**Setup:** Linux, an RTX A4000 (nvidia-smi index 0) and an RTX 5070 Ti (index 1), driver 580.159.04, CUDA 13.0, Qwen3.8-Flash-Next IQ3_S. Two configs, both run with `python tools/strata_bench.py --config <cfg> --think --runs 2`:

- solo: `"gpu": [1]`. The server runs on the 5070 Ti only (per-GPU memory: 14.1 GB on the 5070 Ti, the A4000 idle).
- helper: the 5070 Ti with the A4000 as expert helper (`--expert-cache-device1 --remote-expert-opt`).

**What the report says, for both runs:**

- folder `2026-10-12-community-2x-rtx-a4000-qwen3-8-flash-next-iq3-s`: two A4000s, which neither run had;
- title `Community benchmark on NVIDIA RTX A4000 + NVIDIA GeForce RTX 5070 Ti` and `GPU and VRAM: ... 32 GiB VRAM`, even for the solo run that used one 16 GB card.

**Cause, as far as I can read it:**

1. `gpu_slug()` slugs `names[0]` and prefixes `len(names)x-`, so mixed cards collapse into N copies of the first one.
2. `gpu_names` comes from `gpu_info_local()`, which lists every GPU of the host (nvidia-smi), not the GPUs the server was started on (the config's `gpu` list or `--gpu`).

**Expected:** the GPUs the server actually used, named one by one when they differ, e.g. `rtx-5070-ti` for the solo run and `rtx-5070-ti-plus-rtx-a4000` for the helper run.

This matters for the shared format proposed in #713: a mixed-GPU or partial-GPU report would be filed under the wrong hardware.

本站相关内容

相关页面的快捷入口。