Pull requests / #2613

#2613 Add RTX 3090 Ryzen 5950X IQ3_S benchmark

open · @flameflame1337 · 0 comentarios · En GitHub

BenchmarksAMD / HIPNVIDIA / CUDAModels & quantsDocumentationWindows

Descripción

<html>
<body>
<!--StartFragment--><html><head></head><body><h2><span>Summary</span></h2><p><span>Adding community benchmark results for Qwen3.8-Flash-Next IQ3_S running on Strata with an NVIDIA RTX 3090 and AMD Ryzen 9 5950X.</span></p><h2><span>Hardware</span></h2><ul><li><p><strong><span>GPU:</span></strong><span> NVIDIA RTX 3090 (24 GB VRAM)</span></p></li><li><p><strong><span>CPU:</span></strong><span> AMD Ryzen 9 5950X (16 cores / 32 threads)</span></p></li><li><p><strong><span>RAM:</span></strong><span> 128 GB DDR4</span></p></li><li><p><strong><span>PCIe:</span></strong><span> 4.0 x16</span></p></li><li><p><strong><span>OS:</span></strong><span> Windows 11</span></p></li></ul><h2><span>Configuration</span></h2><ul><li><p><strong><span>Model:</span></strong><span> Qwen3.8-Flash-Next</span></p></li><li><p><strong><span>Quantization:</span></strong><span> IQ3_S</span></p></li><li><p><strong><span>Context length:</span></strong><span> 131,072 tokens</span></p></li><li><p><strong><span>KV cache:</span></strong><span> INT8 with RAM streaming</span></p></li><li><p><strong><span>Vision:</span></strong><span> Enabled</span></p></li><li><p><strong><span>Calibration:</span></strong><span> Automatic</span></p></li></ul><h2><span>Benchmark Results</span></h2>
Metric | Median
-- | --
Chat decode | 95.00 tok/s
Chat first token | 0.36 s
4K prompt processing | 2,211.30 tok/s
4K first token | 1.86 s
16K prompt processing | 1,781.20 tok/s
16K first token | 9.07 s
Agent cache hit | 68.62%
Agent turn first token | 2.04 s
Agent decode | 94.40 tok/s

<h2><span>What changed</span></h2><p><span>Added the automatically generated community benchmark report containing:</span></p><ul><li><p><code><span>README.md</span></code><span> — benchmark summary</span></p></li><li><p><code><span>results.json</span></code><span> — detailed benchmark data</span></p></li><li><p><code><span>engine-log-excerpt.txt</span></code><span> — engine log excerpt</span></p></li></ul><h2><span>Extra Notes</span></h2><p><span>The model loaded and ran successfully on this configuration.</span></p><p><span>Thanks for developing Strata and making large local AI models accessible on consumer hardware!</span></p></body></html><!--EndFragment-->
</body>
</html>

En el sitio

Enlaces a install, modelos, releases.