反馈 / #2378

#2378 setup --calibrate dies with 'there is no GPU 0,1' on a dual-GPU config using the string gpu form

open · @MEsfits · 1 评论 · 去 GitHub 看

Setup & installServer & APIMulti-GPUAMD / HIPNVIDIA / CUDA

说明

## What happens

`./setup.sh --calibrate` on a dual-GPU config whose JSON carries `"gpu": "0,1"` (the string form) runs all measurement rounds to completion, then dies at the save step:

```
[X]  there is no GPU 0,1: 0 = NVIDIA GeForce RTX 3090 Ti, 1 = NVIDIA GeForce RTX 3090
Setup stopped. Fix the item above and run it again - everything already done is kept and skipped.
```

Repro: any dual config with `"gpu": "0,1"` + `"layer_split": "auto"` (e.g. hand-cloned from a setup-written one, or any config where the value stayed a string). Single-GPU and list-form (`[0, 1]`) configs save fine.

## Root cause

`hardware_key()` in `setup.py` passes the raw config value to `gpu_info()`, which matches one card by number:

```python
gl = [gpu_info(i) or {} for i in sel] if isinstance(sel, list) else [gpu_info(sel) or {}]
```

A string `"0,1"` is not a list, so it is looked up whole as one GPU id and matches nothing. Note the server (`serve/server.py:gpu_list()`) already splits the string form, so configs like this start and serve fine — only setup's save path chokes.

Same class of bug one function down in `recommend_remote_expert_opt()`: `if not isinstance(cfg.get("gpu"), list) ...` silently skips the helper-cache opt for string-form dual configs instead of applying it like the list form.

## Collateral damage (worth fixing alongside)

The crash lands *after* a partial config write. Observed on the victim file: tier-measurement leftovers baked in (`--adapt-every 1 --adapt-swaps 160 --adapt-decay 0.97`) while previously tuned flags were wiped (`--pcie-frac` gone, `--spec-min-p` back to default). A failed save should leave the user's config byte-identical — write atomically (temp + rename) or not at all.

## Suggested fix

Normalize once, at the boundary (verified locally against this repo):

```python
def gpu_sel(value) -> list:
    """A config's "gpu" as a list of card numbers: 0, [0, 2] or "0,2"."""
    if value is None or value == "":
        return []
    items = value if isinstance(value, (list, tuple)) else str(value).split(",")
    sel = []
    for x in items:
        x = str(x).strip()
        if x == "":
            continue
        sel.append(int(x) if x.isdigit() else x)
    return sel
```

- `hardware_key()`: `sel = cfg.get("gpu"); if isinstance(sel, str): sel = gpu_sel(sel)` (keep `None` flowing to `hip_config_cards`, where it means "default card" — normalizing it to `[]` breaks `test_no_gpu_in_the_config_is_the_supported_card_with_the_most_vram`, found that the hard way).
- `recommend_remote_expert_opt()`: `if len(gpu_sel(cfg.get("gpu"))) < 2: return`.

Verified: string `"0,1"` now yields the identical hardware key as `[0, 1]`; single-int, single-string, and missing-key forms unchanged; all 36 `test_setup_amd.py` tests pass; full `test_setup_*.py` run (583 tests) shows only the same 10 pre-existing environment failures as unmodified code (confirmed via `git stash` A/B).

Happy to open this as a PR if you want the patch as-is.

本站相关内容

相关页面的快捷入口。