Pull requests / #2387

#2387 bench: community report 2026-10-13 CPU-only Swift 1.5 IQ2_XS (Ryzen 9 3900X)

open · @monscoro · 0 Kommentare · Auf GitHub

BenchmarksSetup & installAMD / HIPModels & quantsDocumentationWindows

Beschreibung

Adds `bench/results/2026-10-13-community-cpu-swift-1-5-iq2-xs/` per docs/COMMUNITY_BENCHMARKS.md. Results-only submission: no engine or setup changes.

Hardware and software
- CPU only (no GPU): AMD Ryzen 9 3900X (12 cores, 24 threads), 63.9 GiB RAM, Windows 11 (10.0.26200)
- Strata project 0.1.43.1, commit a0116b27, engine 0.1.43.1, install's own configuration, hip backend

Model and configuration
- swift-1.5-iq2_xs (Swift-Qwen3.8-Flash-Next-GSQ-RCO-abliterated IQ2_XS), 262,144-token context
- --expert-cache auto --prefill auto --spec 4 --spec-min-p 0.5 --mtp rt --kv int8 --kv-resident 32768

Method
- tools/strata_bench.py suite, 3 runs per test, medians with ranges; model loading not timed; one warm-up chat and a 1K prompt discarded
- short chat, fresh 4K and 16K prompts, and an agent session (system prompt, four tools, five turns that each add about 1500 words)

Results (medians with ranges)
- short chat: decode 37.9 tok/s (34.5-41.9), TTFT 0.84 s (0.74-1.00)
- 4K fresh prompt: prompt 455.6 tok/s (453.2-459.8), TTFT 8.86 s (8.78-8.91)
- 16K fresh prompt: prompt 542.8 tok/s (542.0-580.9), TTFT 29.63 s (27.69-29.68)
- agent session follow-up turns: decode 43.0 tok/s (33.1-56.1), conversation cache hits 68.6 %

Limitations
- Speed only; no correctness check was run
- Background workloads, power limits, storage and PCIe link not recorded

Mehr auf der Site

Links zu Install, Modellen, Releases.