Pull requests / #2721
#2721 prefill: STRATA_PF_PEER_CPU=1 (opt-in) runs the CPU share beside a peer GPU (--peer-device)
open · @bulhenvaster · 0 comentários · No GitHub
AMD / HIPNVIDIA / CUDAModels & quantsLinux
Descrição
The share took nothing there (cpu_maybe required !m.pp). With the switch it takes only experts that neither card computes (not peer-held, not peer-streamed), and its rows go before the peer's block, so the peer's block stays the tail the peer combine reads from back_at (layout: primary GPU, CPU, peer). The staged chunks then go up to cpu_share_max()'s 3072 tokens instead of the default share's 1024 (an explicit STRATA_PREFILL_CPU_SHARE_MAX still wins). Without the switch nothing changes. RTX 5060 Ti (PCIe 3.0 x8) + RTX 3060 (x16), Xeon E5-2683 v4, IQ3_XXS, a 161-173K-token agent conversation, tool results read one after another (STRATA_PREFILL_TIMING): 117-132 tokens 1.80-1.92 -> 1.16 s, 325-335 2.12-2.89 -> 1.36-1.63 s, 524 3.33 -> 1.99 s, 1067 3.74 -> 2.64 s, 1170 3.81 -> 3.26 s, 2012 4.33 -> 4.22 s, 3.2-3.5K unchanged; the chain of 12 reads 36.0 -> 29.7 s (the base twice: 36.1, 36.0). A question about the last tool output answered the same in all three runs. In daily use on that rig since 2026-10-15. Not tested: HIP, SYCL, a layer split's helper (the change is in the CUDA prompt path's host code; the share stays off on HIP unless set), other peer pairs. **How it was checked.** Built on its own on top of `main` (296b59d2): CUDA 13.3, sm_86 + sm_120, Linux (Ubuntu), the `strata` target. HIP, SYCL and Metal were not built: I have only the NVIDIA rig. One change per PR; nothing else from my fork is in it. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
No site
Links install, modelos, releases.