Pull requests / #2863

#2863 docs(bench): add RTX 5090 CUDA 13.2.2 community results

open · @William-AIdev · 0 comentarios · En GitHub

BenchmarksNVIDIA / CUDAModels & quantsDocumentation

Descripción

Adds a community speed report for RTX 5090 32 GB + Ryzen 7 7700X + 64 GB RAM, PCIe Gen5 x8, running IQ3_S with a CUDA 13.2.2 source build of commit 296b59d2.

The saved configuration uses a 262,144-token context limit, MMQ enabled, int8 KV, and official calibration results. Includes the official suite's README and per-request JSON: three runs per test with fresh 4K/16K/32K inputs, short chats, and multi-turn agent sessions. Median prefill is approximately 3,387 / 6,352 / 6,483 token/s; short-chat decode is 154.9 token/s.

Validation: parsed JSON, checked summary values against the generated report, and scanned both files for credentials and machine-specific paths. Removed the remaining library path and bind address; measurements are unchanged. This is a results-only contribution; no engine or default changes. It is not a current-main performance comparison, correctness evaluation, or full-256K-input test.

Related investigation: #2685. The MMQ/tool-calling follow-up is posted separately there.

En el sitio

Enlaces a install, modelos, releases.