#1977 Community benchmark 2026-10-11: RTX 5070 Ti (PCIe x8 dock) and RX 7900 XTX (Windows, eGPU over USB4) - 0.1.43, A/B of the new defaults and the #1630 prefill cap

Beschreibung

# Community benchmarks: RTX 5070 Ti (PCIe x8 dock) and RX 7900 XTX (Windows, eGPU over USB4) — engine 0.1.43, with A/B runs

Four `tools/strata_bench.py --long` runs on one machine on 2026-10-11, engine 0.1.43, each arm three runs, medians with ranges. Two installations sharing one host, one card at a time. Both cards are in docks, and the link widths matter — flagged per COMMUNITY_BENCHMARKS.md.

## The machine

- Beelink GTi14 mini PC: Intel Core Ultra 9 185H (6P+8E, 22 threads, AVX2, no AVX-512), 96 GB DDR5, NVMe, Windows 11 (10.0.26300).
- **RTX 5070 Ti 16 GB** in the Beelink EX Pro expansion dock — **PCIe 5.0 x8** (the dock's link width by design; engine probe 25.5 GB/s host-to-device). Drives four displays (~2.6 GB VRAM held by the desktop). Driver 617.42, CUDA 13.0 build.
- **RX 7900 XTX 24 GB** (gfx1100) in an AOOSTAR AG02 dock (ASM2464PDX) on a **Thunderbolt 4 / USB4 port**, x16 slot inside the dock. No displays on it. Driver 32.0.31041.1004, HIP build (bundled ROCm 10.2.0a20260930, hipBLASLt 100500 with the gfx1100 tuning table).
- Model: Qwen3.8-Flash-Next IQ3_XXS, 262144-token context, one slot. Sources: upstream `v0.1.43` (`7695859a`) + a local config layer (deviations tabled in each README); no engine source changes.

## 1. RTX 5070 Ti (Windows): the 0.1.42/43 speed defaults measure larger here than the published tail-skip figure

Baseline (defaults on: tail skip 7, measured PCIe share, CPU prompt share) vs the same day with all three opted out (`STRATA_ROUTE_TAIL_SKIP=0`, `STRATA_PCIE_FRAC_DEFAULT=old`, `STRATA_PREFILL_CPU_SHARE=0`):

| Metric | defaults on | defaults off | change |
|---|---:|---:|---:|
| chat decode tok/s | 63.0 (52.4-68.6) | 48.8 (38.3-58.4) | **−22.5%** when opted out |
| chat first token (s) | 0.60 | 0.75 | **+25.3% slower** when opted out |
| agent decode tok/s | 72.3 (43.0-94.0) | 57.7 (30.8-74.7) | **−20.2%** when opted out |
| 4K / 16K / 32K prompt tok/s | 1614.7 / 2383.0 / 2384.8 | 1607.9 / 2452.8 / 2429.4 | within noise (±3%) |

The defaults are worth **+29% chat decode and +25% agent decode** on this card (ranges do not overlap), with the prompt side unchanged. Published +13-16% was tail skip alone; the three defaults together measure larger. Attribution per default was not done (one policy switch, three env vars). Answer drift not measured in these runs (speed-only suite); the published KL figures apply to the tail-skip component.

## 2. RX 7900 XTX (Windows, eGPU over USB4): the `--prefill auto` cap (#1630) costs ~24% prompt here, not 14%

Baseline with explicit `--prefill 8192` vs the same day with stock `auto` (engine caps at 6144 on Windows AMD). One variable; everything else identical:

| Metric | 8192 | auto → 6144 | change |
|---|---:|---:|---:|
| 16K prompt tok/s | 308.6 | 236.4 | **−23.4%** |
| 32K prompt tok/s | 322.0 | 244.0 | **−24.2%** |
| 16K / 32K first token (s) | 52.1 / 99.6 | 68.0 / 131.4 | +31% / +32% |
| 4K prompt tok/s | 174.3 | 175.8 | unchanged (chunks under 6144, as expected) |
| chat decode tok/s | 45.6 | 44.3 | −2.9%, ranges overlap |
| agent decode tok/s | 74.9 | 71.9 | −4.0% |
| agent re-read tokens/session | 4 | 144 (144-145 in all 3 runs) | chunk boundaries at 6144 misalign with turn boundaries |

The cap costs more prompt speed than the published −14% (645 vs 748 tok/s, internal card) — consistent with expert streaming being the bottleneck and USB4 being narrower than internal PCIe. Decode is essentially unchanged: the smaller RAM complement at 6144 (25.86 vs 26.61 GiB, −455 experts) did not show up as a measurable decode cost.

Question: would a RAM-aware cap (or a documented "explicit 8192 is fine above ~64 GB RAM") be reasonable? This box ran 8192 since 0.1.41 without incident; commit headroom at start was 75.6 GiB of a 143.7 GiB limit. Safety on smaller-RAM Windows AMD boxes was not tested.

## 3. Tool note: `strata_bench.py` mislabels the GPU on a dual-GPU machine

The local GPU probe reads `nvidia-smi` first, so both AMD runs were written as "NVIDIA GeForce RTX 5070 Ti" in the report header, folder name, and `results.json` (`machine.gpu`, `gpu_vram_mib`, `gpu_driver`), even though `--url` pointed at the AMD server. Fixed by hand in the attached files (README headers, folder names, `results.json` fields; `machine.engine_info` was correct — it comes from the server). Suggest: when `--url` is used, prefer the GPU the server reports (or probe `rocm-smi` too).

## What was not tested

- Correctness / answer drift in these runs (speed suite only).
- Per-default attribution on NVIDIA.
- Whether 8192 is safe on smaller-RAM Windows AMD boxes.
- `--clients` concurrency, long soaks, Coder/Swift variants.

Full per-request data and settings tables are in the four README folders (sanitized; local paths, names and keys checked to be absent):

- [NVD baseline (defaults on)](https://gist.github.com/vitaliso/64ef969dfc677087c900868250bcad2a)
- [NVD arm B (defaults opted out)](https://gist.github.com/vitaliso/553ad832de6e1aba81c449cdef1916f6)
- [AMD baseline (--prefill 8192)](https://gist.github.com/vitaliso/3ce6dd3ff072174e8ce6d343019c50e6)
- [AMD arm B (--prefill auto, 6144 cap)](https://gist.github.com/vitaliso/afe909c444f190a53856ec08db869dfc)