Pull requests / #2953
#2953 fix(serve): reject duplicate GPUs in automatic replica groups
open · @j-luwierski · 0 comentários · No GitHub
Setup & installServer & APIMulti-GPUAMD / HIPNVIDIA / CUDALinux
Descrição
## Summary
With a numeric replica count, `gpu_groups()` returns before checking for duplicate GPUs. For example, `{"gpu":[0,0],"replicas":2}` produces `[[0],[0]]`; `[0,1,1,2]` with two replicas produces `[[0,1],[1,2]]`. Startup can therefore launch independent engines on overlapping cards. Equivalent explicit groups are already rejected.
Send automatically derived groups through the same existing validation as explicit groups, so duplicate cards raise `ReplicaConfigError` before replica engines start.
## What changed
- `serve/replicas.py`: replace the numeric branch's early return with group construction, then use the existing shared validation.
- `serve/test_replicas.py`: cover duplicates across and within derived groups, list and comma-separated GPU inputs, normalized duplicate IDs (`0,00`), existing explicit-group rejection, preserved card order and disabled replicas.
Valid automatic and explicit groups keep their order and behavior. Omitted/false/zero/one replica counts keep the existing no-replica path. No engine, routing, sampling, setup-default or backend code changes.
## Validation
Linux, Python 3.13, upstream base `477ac13cbe8e582c20484871ce459d20d82df09d`:
- New automatic-group regression against the original code: all six duplicate cases fail to raise. They pass after the fix.
- `python -m unittest serve.test_replicas serve.test_parallel -q`: **51 tests pass**, 53.1 s, using the existing repository Python environment. Includes fake-engine integration tests; no model or GPU needed.
- Compared original and changed `gpu_groups()` over every GPU sequence of length 0..4 from IDs 0..3, in list/string form, with replica counts None/false/0/1/2/3/4: **3758 outcomes identical; 1016 previously accepted duplicate-GPU cases now rejected**. These are synthetic configuration checks, not hardware measurements.
- `git diff --check` passes.
No multi-GPU inference, memory-pressure or performance measurement was performed. No CUDA/HIP/SYCL source or build files are touched; backend builds were not run.
## Related work
Searched open issues/PRs for replicas, duplicate GPUs and `gpu_groups`; no matching automatic-group validation fix found. #2886 concerns replica pipe ownership/statistics rather than GPU grouping.No site
Links install, modelos, releases.