Pull requests / #852
#852 engine: --gpu selects the visible GPUs without environment variables
closed · @anon761 · 0 评论 · 在 GitHub 查看
Server & APIMulti-GPUNVIDIA / CUDA
描述
## What A `--gpu LIST` option that sets `CUDA_VISIBLE_DEVICES` and `CUDA_DEVICE_ORDER=PCI_BUS_ID` **inside the engine**, at the top of `main`, before anything touches CUDA. ## Why `CUDA_VISIBLE_DEVICES` is read at the first CUDA call, so every caller that wants to pin an engine to specific cards has to export it before the process starts (a run script, a config-driven launcher, a CLI export). With `--gpu`, the same start line works with **no environment variables at all**: ``` strata --serve --gpu 0,1 --layer-split auto --pack … --native … ``` `LIST` is comma-separated nvidia-smi/PCI indices — the numbering `server.py`'s config `"gpu"` already uses; `CUDA_DEVICE_ORDER=PCI_BUS_ID` keeps CUDA from renumbering the cards fastest-first, matching `server.py`'s `child_env`. The option is consumed by the normal option loop and documented in `usage()`. No behaviour change when `--gpu` is absent.
站内延伸阅读
链到安装、模型与版本说明,便于 SEO/GEO,非官方 issue 正文。