Pull requests / #2223

#2223 verify: STRATA_NO_PCIE_GROUP=1 drops the empty PCIe-share stage with --pcie-frac 0

open · @rafatxf · 0 comentários · No GitHub

Setup & installServer & APINVIDIA / CUDAModels & quants

Descrição

With `--pcie-frac 0` the pool never plans PCIe experts, yet every layer of a verify window still runs the PCIe-share stage over zero groups: the flag B wait, `fetch_blobs` + `rebase_ptrs` (staging mode 2) and the grouped gate/up, SwiGLU and down launches with `kPcieGroupRows` rows that find nothing to do. Six dependent kernels, ~18 µs a layer on an RTX 2080 Ti (kernel-node nsys), ~0.86 ms of a ~25 ms window.

`STRATA_NO_PCIE_GROUP=1` records the window graph without that stage. It is opt-in and only valid with `--pcie-frac 0` (the graphs are captured once, and the share can otherwise be refined at run time): if the pool ever publishes a plan with PCIe experts, the host loop stops the engine with a message (`STRATA_NO_PCIE_GROUP=1 needs --pcie-frac 0 (layer L planned N PCIe experts)`) after releasing the GPU's waits, instead of letting the GPU skip experts. I checked that path by starting the engine without `--pcie-frac 0`: it stops at the first window that plans a PCIe expert.

On this card, with the expert cache warm (~96-97 % VRAM hits in decode), the CPU takes the few misses the PCIe share would have taken without a measurable loss: `--pcie-frac 0` alone was neutral (Spanish 86.9 vs 87.8 t/s, code 106.3 vs 106.1, 32K 78.2 vs 78.4 - within the run-to-run spread).

## Setup

RTX 2080 Ti 22 GB (sm_75), i7-6700 (4 cores), 56 GB DDR4-2133, PCIe 3.0 x16, CUDA 13.0, SM 1650 MHz, 240 W. Flash-Next, canonical Q2_0 pack, `--resident-experts`, `--max-context 524288`, `--serve`, with the other open Turing decode PRs applied.

## Engine (512 new tokens, two runs each)

| | `--pcie-frac` auto | `--pcie-frac 0` + `STRATA_NO_PCIE_GROUP=1` | |
|---|---:|---:|---:|
| Spanish chat (57-token prompt) | 87.8 t/s | 89.8 t/s | +2.3 % |
| code | 106.1 | 108.4 | +2.2 % |
| English, 32K | 78.4 | 80.1 | +2.2 % |
| English, 128K | 75.0 | 76.8 | +2.4 % |
| English, 389,560 tokens | 74.7 | 75.1 | +0.6 % |

The prompt reads are unchanged (32K 17.6 s, 390K 223.6 s).

A follow-up could make this automatic when `--pcie-frac 0` is given (no env), if you prefer that.

Part of the Turing set tracked in #2197.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

No site

Links install, modelos, releases.