改动 / #2607
#2607 serve: the card-order line says how to try the engine's suggested split point in the same order (#2085)
closed · @qiuzan2001 · 0 评论 · 去 GitHub 看
Server & APIMulti-GPUDocumentationLinux
说明
On #2085 (found while reading that report's `--layer-split` planner notes).
**What it changes.** With `"layer_split": "auto"`, `serve/server.py` `ordered_gpus()` puts the faster card last
(#1352), so the engine gets the cards as, say, `2,0`, and its NEAR TIE line (`src/program/generate.cpp` ~4320,
"To try the other: --layer-split K") counts K in that order. A manual `layer_split` keeps the order the config lists
(`0,2`), so a user who copied K got its layers on the other cards, with the slower card running the head. The
server's `layer split: card order 2,0 (...)` line now ends with `to try the engine's --layer-split K in this order,
list "gpu": [2, 0] with it, #2085`, and docs/MULTI_GPU.md ("Card order with auto") says the same.
**Which fix and why.** The fix is the hint only, not reordering or translating K:
- Reordering a manual split as well would move the layers of every hand-tuned split that exists today.
- Translating K for two cards (K' = layers - K in the listed order) gives the same layer counts, but the head,
draft and verify then run on the slower card, so it is not the placement the engine priced.
- A manual split keeps the listed order (it returns before the reorder, the #1576 check included), so
`"gpu": [2, 0]` with `"layer_split": "K"` gives the engine the same device order and K.
**What it leaves alone.** Every card order and every split (no placement changes). The line is printed only when
the order changes, as before.
**How it was checked.** `serve/test_server.py` `GpuChoice.test_card_order`: the reorder line names `"gpu": [2, 0]`
and the engine's suggestion, and a config listing `[2, 0]` with a manual K gets `[2, 0]` (fails on `main`, passes).
With the venv's Python, `serve.test_server`: OK. Python 3.14, Ubuntu (Linux 7.0); no GPU, no engine run.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
本站相关内容
相关页面的快捷入口。