Issues / #2887

#2887 strata_bench.py records a layer-split run as single-GPU (card list and VRAM total)

open · @kristoffern · 0 comments · View on GitHub

BenchmarksMulti-GPUNVIDIA / CUDADocumentationLinux

Description

### What happened

A benchmark of a two-card layer split (RTX 3090 + RTX 4080 SUPER) produces a report naming only the first card:

```json
"gpu": ["NVIDIA GeForce RTX 3090"], "gpu_vram_mib": 24576
```

while the server reports both correctly:

```json
"machine": {"gpu": {"name": "NVIDIA GeForce RTX 3090 + NVIDIA GeForce RTX 4080 SUPER",
                    "total_mib": 40952}}
```

and the engine log says `layer split: layers 0-28 (CUDA0), 29-47 (CUDA1), one hand-off per window`.

Cause, `tools/strata_bench.py:614-616`:

```python
elif len(used) > 1 and static.get("gpu_name"):
    named = [g for g in used if g["name"] == static["gpu_name"]]
    used = named[:1] or used[:1]
```

With a split, `static["gpu_name"]` is `"A + B"`, so no single card equals it, `named` is empty, and `used` is truncated to the first card. `gpu_vram_mib` is then summed over that one card (lines 627-628), overriding the server's correct total. The `" + "` form is already parsed 20 lines below (#2001), but that branch only fires when the served cards are *not* all local (the other-vendor case), so a split across two local NVIDIA cards never reaches it.

The branch is only reached when `cfg` is None, i.e. with `--url`. On current main that is the only way to benchmark at all, because `bench.sh` crashes when it starts the server itself (#2872). So at the moment every layer-split run benchmarked with the tool is reported as single-GPU, in both the card list and the VRAM total.

To reproduce: a config with `"gpu": [0, 1]`, start the server, then `./bench.sh --url http://127.0.0.1:<port>`. The generated `README.md` and `results.json` name one card.

Suggested fix: match each name the server gives, keeping the count.

```python
elif len(used) > 1 and static.get("gpu_name"):
    served_names = [n for n in static["gpu_name"].split(" + ") if n and n != "?"]
    named = [g for g in used if g["name"] in served_names]
    used = named[:len(served_names)] or used[:1]
```

Checked against this machine's values: the split case becomes `["RTX 3090", "RTX 4080 SUPER"]` / 40952 MiB, and both single-card-of-two cases are unchanged.

### Strata version, GPU, OS

Engine 0.1.45, main at 477ac13c (these lines unchanged on main today). RTX 3090 + RTX 4080 SUPER, Linux (Ubuntu 24.04 base), engine compiled from source.

Related on this site

Quick links to related pages.