改动 / #2404

#2404 Community benchmark: RTX 4080 SUPER 16 GB, Ryzen 7 7800X3D, 64 GB, IQ3_S at 192K (0.1.44.1)

open · @olituks · 0 评论 · 去 GitHub 看

BenchmarksSetup & installNVIDIA / CUDAModels & quantsDocumentationWindows

说明

Results only, no code changes.

- **Hardware:** RTX 4080 SUPER 16 GB (PCIe 4.0 x16), Ryzen 7 7800X3D, 64 GB RAM, Lexar NM790 4 TB NVMe, Windows 11, driver 616.92 (CUDA 13.4)
- **Strata:** 0.1.44.1 (296b59d2), official Windows engine
- **Model:** original Qwen3.8-Flash-Next IQ3_S, 196,608-token context, KV q4_0 with `--kv-grow`, MTP on, vision on the CPU
- **Settings:** calibrated with `--calibrate`, no opt-in env variables
- **Suite:** `BENCH.bat --long` (chat, 4K/16K/32K fresh prompts, agent session), 3 runs each

Medians: chat decode 88.6 tok/s, agent decode 98.1 tok/s, prompt 2,165 / 3,456 / 3,489 tok/s at 4K / 16K / 32K.

The README also has a table of six configurations measured the same day, from the install defaults to this one.
Moving the image encoder from the GPU to the CPU was by far the largest change (+35% chat decode, +44% agent decode,
+62% prompt at 16K/32K). That is much more than the 2-8% DETAILS.md describes, so I opened a separate issue: [TODO: link].

Limitations: speed suite only, no needle or quality test; decode at long context is only checked by one real session
(~90K tokens, 97 tok/s), not by the suite.

本站相关内容

相关页面的快捷入口。