Pull requests / #1225
#1225 bench: add R9 dual RTX 2080 Ti community results
closed · @cezartolvai · 0 comentários · No GitHub
BenchmarksMulti-GPUNVIDIA / CUDAModels & quants
Descrição
## Summary Adds a results-only community benchmark report for a Ryzen 9 5950X with 2x RTX 2080 Ti (SM 75), using Flash-Next GSQ-RCO IQ3_XXS at 131K context and an automatic two-GPU layer split. - 3 fresh-prompt runs at 4K and 32K, each with 256 greedy output tokens - Raw streamed responses, requests, engine metrics, and aggregate JSON included - No model files, packs, credentials, or engine changes ## Median results | Prompt | Prompt tok/s | Decode tok/s | | ---: | ---: | ---: | | 4,096 | 411.1 [404.0-413.9] | 52.8 [52.6-54.2] | | 32,768 | 477.2 [435.0-574.0] | 42.5 [42.5-42.8] | The report retains the 32K prefill variation and documents limitations: no 128K, recall, vision, thermal, or concurrency runs.
No site
Links install, modelos, releases.