Pull requests / #2471
#2471 bench: community report - RTX 3090 + RTX 5060 Ti expert tier, UD-IQ4_XS, custom-line 0.1.43.1 engine
open · @txiki739 · 0 comments · View on GitHub
BenchmarksServer & APINVIDIA / CUDAModels & quantsDocumentation
Description
Results-only community report: `bench/results/2026-10-14-community-rtx-3090-plus-rtx-5060-ti/` (README in the documented template, `results.json` with every request, `needles.json`, a scrubbed copy of the configuration and a trimmed engine start excerpt). **Hardware:** RTX 3090 24 GiB (main card) + RTX 5060 Ti 16 GiB (expert tier, `--second-gpu 0`), Ryzen 7 5700X (no AVX-512), 125.7 GiB RAM, NVMe; driver 615.78.08; both cards at power limits below their defaults (250 W / 150 W). **Engine and server:** engine 0.1.43.1, source build of the `eddoursul/Strata` `custom` line at `c4bcb6e8` (no engine-code change over it locally; the local commits touch configuration and measurement files only); server: this repo's 0.1.44.1 `serve/server.py`. Model: Unsloth **UD-IQ4_XS** with a native-experts pack, 200,192-token context, int8 KV with the elastic grow, prefill 32,768, `--spec 4`, adaptive tier (`--adapt-swaps 192`); `STRATA_IQ256_GATHER=1`, `STRATA_POOL_FUSED=1`. **Measured** (medians over 3 runs, `tools/strata_bench.py --long`): prompt **1,723 tok/s at 4K**, **2,424 at 16K**, **2,418 at 32K**; chat decode 100.1 tok/s; agent turns 124.1 tok/s with 68.8% conversation-cache reuse; needle recall **6/6** at 32K and 128K (`tools/needle_bench.py`). **Limitations (also in the README):** one machine, one engine session with warm caches (model loading not timed); power limits below defaults; thinking off for the suite; concurrency not tested (one parallel slot). The `results.json` machine block was completed and scrubbed by hand on top of the tool's own redaction (both cards, CUDA, storage, power limits; a local host name and a private DNS name and the private model alias removed). `COMMUNITY.md` rows and the docs list are left to the maintainers per the doc. No engine changes are in this PR.
Related on this site
Quick links to related pages.