Pull requests / #2428
#2428 bench: RTX 5060 Ti 16 GB on Ubuntu 24.04, 0.1.44.1, Flash-Next IQ3_XXS
open · @liulib · 0 コメント · GitHub で見る
BenchmarksSetup & installNVIDIA / CUDAModels & quantsDocumentation
本文
Results-only submission: one folder under `bench/results/`, no engine changes. **Hardware:** RTX 5060 Ti 16 GB, Ryzen 7 5700X, 64 GB DDR4, B550M, Acer N3500CN NVMe (PCIe 3.0); Ubuntu 24.04.5 Server (headless), driver 580.178.04 open, CUDA 13.0. **Strata / model:** 0.1.44.1 release source zip, engine built locally by `setup.sh` (sm_120); Flash-Next IQ3_XXS, 262,144-token context, KV int8 with `--kv-resident 32768`, conversation cache 6144 MiB × 4 slots, vision (CPU mmproj) loaded. **Configuration tested:** calibrated settings (`--pcie-frac 0.20 --spec-min-p 0.50 --adapt-every 1 --adapt-swaps 80 --adapt-decay 0.97`) plus `STRATA_ADAPT_OVERLAP=1`. Full argument list in the README. **Results (medians, `tools/strata_bench.py`, 3 runs):** chat decode 74.6 tok/s, TTFT 0.45 s; 4K prompt 1,321 tok/s; 16K prompt 1,835 tok/s; agent session decode 83.7 tok/s, turn TTFT 3.25 s, 73 earlier-turn tokens re-read per session. **Limitations:** speed only (no correctness checks); the suite ran against an already running server, so `engine-log-excerpt.txt` is absent and the settings were added to the README by hand; `--long` and `--clients` not run. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
関連リンク
インストール・モデル・リリースへの站内リンク。