Pull requests / #450
#450 calibrate: a gpu list is not a layer split (#447)
closed · @EdderTalmor · 0 Kommentare · Auf GitHub
Setup & installServer & APIMulti-GPUNVIDIA / CUDADocumentation
Beschreibung
Fixes #447. `tools/calibrate.py` read "the config has a `gpu` list" as "this model splits layers across several cards" and appended `--layer-split auto`. A `gpu` list does not imply a split: the helper-GPU expert tier (`--expert-cache-deviceN`, docs/SECOND_GPU.md) runs one stage on one card with a second card visible for the tier, and the engine rejects `--expert-cache-remote` together with a layer split — so `--calibrate` exited before the sweep started and left the machine uncalibrated with only a warning line. The fix mirrors `serve.server.engine_args()`: append the split only when several cards actually run stages (`len(gpu_list(cfg)) > 1`). Extracted as `split_args(cfg)` so it is testable without a GPU. Tests added (`tools/test_calibrate.py`): the helper-tier config gets no split, a real multi-card split still gets `auto", an explicit `--layer-split` in the config is left as it is. Full suite passes: 12/12. Verified on the reporter's setup shape: `gpu: [0]` + `CUDA_VISIBLE_DEVICES: "0,1"` + `--expert-cache-device1` — the sweep now starts and writes the calibration record.
Mehr auf der Site
Links zu Install, Modellen, Releases.