#2888 bench: RTX 3090 + RTX 4080 SUPER layer split, with and without other work on the second card

本文

A results-only submission: `bench/results/2026-10-17-community-rtx3090-rtx4080s-split/`. No engine changes.

**Hardware:** RTX 3090 (24 GiB) + RTX 4080 SUPER (16 GiB), both on PCIe 4.0 x8; i9-14900KF (no AVX-512), 123.4 GiB RAM, Linux, driver 595.84. The 4080 SUPER also drives the display (~2.3 GiB), which cannot be removed without ending the desktop session, so it is present in every arm.

**Model:** Qwen3.8-Flash-Next Q2_0, 131072 context, KV int8 with KV streaming, `--spec 4` with the MTP draft layer, otherwise setup's own choices. Engine 0.1.45 (477ac13c), compiled from source.

**Configurations:** four arms on the one machine — each card alone, and the pair as a layer split (`"gpu": [0, 1]`, `"layer_split": "auto"`) in two states. The split arms differ only in whether another CUDA process (a local embedding server, ~3.2 GiB) held memory on the second card, and the `--vram-reserve-later-mib` that left it room (3072 busy, 1536 free).

**Result:** the same pair is **6% slower** at decode than the 3090 alone when the second card is busy, and **17-38% faster** when it is free (chat decode 132.9 vs 113.2 tok/s; 32K prompt 3347 vs 2424 tok/s). The engine log gives the mechanism directly: the second card's expert cache goes from 959 of its 8192 profiled pairs to "82% of the experts resident", and the decode expert-cache hit rate from a 93.0% median to 99.7%. `auto` also moves the boundary from K=32 to K=29.

So the published +18-20% prompt gain for a split reproduces here in direction and size, but only once the second card has room; with a few GiB of someone else's memory on it the same split is a regression. That seemed worth a report, since a second card that does nothing else is not the common case.

**Limitations**, also stated in the README: speed only, no correctness check. One suite run of 3 repetitions per test per arm, so the ranges are run-to-run, not restart-to-restart. The 4080-alone arm carries a 3 GiB reserve for the display, so it is not like-for-like with the 3090's 700 MiB. 4K prompts are slower on the pair in both split arms (the hand-off is overhead before the pipeline fills); the crossover between 4K and 16K was not measured. Long prefill at 16K/32K is near-identical on the two cards alone despite a 34% decode gap, which I did not isolate.

The attached `results-split-*.json` name only the 3090 and 24576 MiB of VRAM: that is #2887, where the bench tool drops the second card on a layer split. The hardware in the README is what actually ran, and `engine-log-excerpt.txt` carries the split lines. `--url` was used throughout because of #2872.

🤖 Generated with [Claude Code](https://claude.com/claude-code)