#2048 bench: 4× V100-16GB Q4_K_XL (sm_70 MMQ + bandwidth-proportional layer split)

Descripción

## Hardware
4× Tesla V100-SXM2-16GB (sm_70), 2× Xeon E5-2696 v4, 247 GiB RAM; asymmetric PCIe x8/x16/x8/x16 (H2D 6.15/11.89/5.90/10.29 GB/s).

## Model & config
Unsloth Qwen3.8-Flash-Next-UD-Q4_K_XL, native-experts pack (--compat-bf16), layer-split 9,25,34 (per-card to PCIe bandwidth), --spec 4, int8 KV, STRATA_SM70_TABLE=1.

## Results
- 2K prefill 458.5 tok/s (median, 5 runs); 81K prefill 2989 tok/s (median, 3 runs)
- Needle recall 6/6 at 32k and 128k (tools/needle_bench.py)

## Main findings
1. -DSTRATA_MMQ_KQUANTS=ON gives ZERO prefill gain on sm_70: the path is confirmed taken (slot borrow 1635 vs 1606), but prefill is unchanged because the bottleneck is PCIe streaming of non-resident experts, not the dequant→FP16 GPU work (the official 4× figure is sm_86 int8-TC only).
2. Bandwidth-proportional layer split (11,25,37 → 9,25,34) = +12.9% prefill and stabilizes repeated 81k requests.

## Limitations
TTFT approximated from engine-log read time; decode single-session; needle covers 32k/128k only.