Pull requests / #665

#665 serve: no automatic --layer-split when the args use --peer-device

closed · @fks · 0 评论 · 在 GitHub 查看

Server & APIMulti-GPUDocumentation

描述

With several GPUs in the config's `"gpu"` the server appends `--layer-split auto`, which the engine refuses beside `--peer-device` (#531). The peer tier could not be started through the server:

```
strata generate: --peer-device cannot be combined with --layer-split (use one or the other)
```

The split (and `--split-skip-if-fits`) is now added only when `--peer-device` is not among the args, and the start line names the peer tier instead. `docs/SECOND_GPU.md` says how to configure it; tests in `serve/test_server.py` (`PeerDevice`), the whole file passes.

站内延伸阅读

链到安装、模型与版本说明,便于 SEO/GEO,非官方 issue 正文。