#3568 [Benchmark] RTX 5090 32 GB - Qwen 3.8 Flash Next Coder IQ1_M, 131K ctx, Vision: ~78 tok/s warm decode

说明

Data point for the benchmark collection (fits the existing RTX-4090/3070 entries).

Hardware: Intel i9-14900KF (16C/32T, AVX2 + AVX-VNNI), RTX 5090 32 GB VRAM, 96 GB RAM, Windows 11 + WSL2, CUDA 13.4, driver 615.78.

Model: Qwen 3.8 Flash Next Coder IQ1_M (MoE, Vision ON), context 131072, KV quant int8, MTP draft layer 835 MiB (--spec 4).

Engine log (warm):
- Expert cache: 11,463 slots = 21.84 GiB VRAM -> 93.3 % of experts resident on GPU; remainder served by 15 CPU workers (8 perf + 4 half-eff cores).
- Expert weights in RAM: 23.42 GiB, loaded at 5.49 GiB/s.
- KV streaming: 32,768 of 131,072 cells in VRAM, rest in 1.55 GiB pinned RAM.
- PCIe probe reported: 36.3 GB/s host->device.
- Vision encoder warm in ~0.9 s.
- Warning seen: "57 MiB of VRAM free - requests may stall" (map is essentially full; --vram-reserve-mib raised afterwards).

Measured decode: 29 tok/s cold (33 tokens), 77.6 tok/s warm (~140 tokens).
Prefill is PCIe-bound (weights streamed from the ~29 GB lookup shard on SSD).

Note: VRAM is exclusive - with a second local server (e.g. LM Studio holding a model) loaded, throughput drops sharply; we unloaded first.