Pull requests / #2665

#2665 docs(MULTI_GPU): the layer split pays most when the CPU pool is the bottleneck

open · @NorwinYu · 0 comentários · No GitHub

BenchmarksMulti-GPUNVIDIA / CUDADocumentation

Descrição

## Summary

One bullet in `docs/MULTI_GPU.md`'s "Which cards and in what order" list, qualifying the
existing "decode is on par with the faster card alone" note. Docs only; no code changes.

## Why

That "on par" conclusion was measured on an RTX 5080 + 3090 over a Ryzen 9 9950X3D — a fast
CPU where the CPU pool is never the thing the window waits on. On a machine where the CPU pool
*is* the bottleneck, the same split is a large decode win, not "on par".

## Measurement

2x identical RTX 2080 Ti 22 GB, dual-socket Xeon E5-2696 v4 (AVX2 only), Strata `main`
@ v0.1.43.1, `unsloth/UD-IQ4_XS` at 131K. A plain layer split (`"gpu": [0, 1]`):

| metric | single | split |
| --- | ---: | ---: |
| decode 424 / 2,944 / 11,582 tok/s | 28.4 / 29.8 / 31.3 | 52.4 / 57.9 / 57.3 |
| window | 70.06 ms | 37.96 ms |
| CPU pool | 9.52 ms | 0.79 ms |

The gain is almost entirely the CPU pool collapsing (9.52 → 0.79 ms/window): when the CPU
pool is what the window waits on, moving half the layers to a second identical card removes
that work.

## Scope

Adds one bullet and a pointer to the new benchmark report
(`bench/results/2026-10-16-community-2x2080ti-dual-socket/`, a separate PR https://github.com/Niko1221/Strata/pull/2662). 

No site

Links install, modelos, releases.