Pull requests / #2560
#2560 docs: MULTI_GPU.md says when STRATA_SPLIT_OWN and STRATA_PREFILL_HELP do not pay (every expert resident, #2461)
closed · @qiuzan2001 · 0 comentarios · En GitHub
Multi-GPUNVIDIA / CUDAModels & quantsDocumentation
Descripción
Closes #2461 (the reporter asked for both sections to carry the precondition). **What it changes.** Two sentences in `docs/MULTI_GPU.md`, with #2461's measurements and its machine (Radeon VII + Radeon Pro VII / Instinct MI50, gfx906, engine 0.1.42, Coder IQ1_M, interleaved A/B, medians): - `STRATA_SPLIT_OWN`: its per-card prompt buffers come out of the expert cache (`no_prefill_borrow` in `generate.cpp`), so where decode needs that cache the trade can lose: the first card's cache fell from 6,889 to 5,353 slots, 4K / 16K prompts read 2.2% / 0.8% faster and decode ran 3.6% / 5.3% slower. `auto` stays off on that box (the buffers are ~3.2 GiB, about 20% of a 16 GB card, over its 12% rule). - `STRATA_PREFILL_HELP`: it hands over only streamed (non-resident) experts (`set_stage_helper` in `src/prefill/prefill.cpp`), so with every expert resident there is little for it to take: 1.5K and 3K prompts within 1.1% of the default, 3 runs each. **What it leaves alone.** The measurements already on the page (R9700 + RX 9070 XT, 2x RTX 3090), both switches and their defaults, and every other line. Nothing says the switches are worse in general. **How it was checked.** The numbers against #2461 line by line; the mechanism and the `auto` rule against the engine source. A sub-agent reviewer checked both; its corrections (the first card's cache, "resident by default", softer wording for PREFILL_HELP) are in. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
En el sitio
Enlaces a install, modelos, releases.