Pull requests / #2665
#2665 docs(MULTI_GPU): the layer split pays most when the CPU pool is the bottleneck
open · @NorwinYu · 0 Kommentare · Auf GitHub
BenchmarksMulti-GPUNVIDIA / CUDADocumentation
Beschreibung
## Summary One bullet in `docs/MULTI_GPU.md`'s "Which cards and in what order" list, qualifying the existing "decode is on par with the faster card alone" note. Docs only; no code changes. ## Why That "on par" conclusion was measured on an RTX 5080 + 3090 over a Ryzen 9 9950X3D — a fast CPU where the CPU pool is never the thing the window waits on. On a machine where the CPU pool *is* the bottleneck, the same split is a large decode win, not "on par". ## Measurement 2x identical RTX 2080 Ti 22 GB, dual-socket Xeon E5-2696 v4 (AVX2 only), Strata `main` @ v0.1.43.1, `unsloth/UD-IQ4_XS` at 131K. A plain layer split (`"gpu": [0, 1]`): | metric | single | split | | --- | ---: | ---: | | decode 424 / 2,944 / 11,582 tok/s | 28.4 / 29.8 / 31.3 | 52.4 / 57.9 / 57.3 | | window | 70.06 ms | 37.96 ms | | CPU pool | 9.52 ms | 0.79 ms | The gain is almost entirely the CPU pool collapsing (9.52 → 0.79 ms/window): when the CPU pool is what the window waits on, moving half the layers to a second identical card removes that work. ## Scope Adds one bullet and a pointer to the new benchmark report (`bench/results/2026-10-16-community-2x2080ti-dual-socket/`, a separate PR https://github.com/Niko1221/Strata/pull/2662).
Mehr auf der Site
Links zu Install, Modellen, Releases.