贡献 / #1784

#1784 bench: add RTX 5060 Ti / Xeon E5-2680 v4 community results

closed · @VykosX · 0 评论 · 去 GitHub 看

BenchmarksSetup & installNVIDIA / CUDAModels & quantsDocumentationWindows

说明

## Summary

Adds a results-only Strata 0.1.41 community benchmark submission for a Windows 10 machine with:

- NVIDIA GeForce RTX 5060 Ti 16 GB
- Intel Xeon E5-2680 v4, 14 cores / 28 threads
- 64 GB RAM
- PCIe Gen 3 x8

Two optimized, text-only Qwen3.8-Flash-Next configurations were measured: Coder IQ1_M and an abliterated Q2_0 derivative.

No engine, server, or Web UI code is changed by this PR.

## Method

The submitted `benchmark.py` is byte-for-byte unchanged from the repository's community benchmark script.

For each model:

- one warm-up request was excluded
- three fresh requests were run serially at 4,096 / 32,768 / 128,000 prompt tokens
- every measured request generated exactly 256 tokens
- temperature was 0
- reasoning was disabled
- every prompt used a unique nonce
- Strata reported **zero reused prompt tokens** in all 18 measured requests
- loading time was excluded; each model remained loaded for its complete nine-request sweep

The compact submission includes results, summaries, initial server status, Strata BUILD metadata, sanitized measured configs, and a README. Bulky raw request/chunk dumps were intentionally omitted and are documented in `TRIMMED.md`.

## Results — Coder IQ1_M

Median of three runs:

| Prompt tokens | Prompt tok/s | Decode tok/s | TTFT |
| ---: | ---: | ---: | ---: |
| 4,096 | **1,179.7** | **47.46** | **3.529 s** |
| 32,768 | **1,600.6** | **51.02** | **20.558 s** |
| 128,000 | **1,859.5** | **49.13** | **69.047 s** |

## Results — abliterated Q2_0

Median of three runs:

| Prompt tokens | Prompt tok/s | Decode tok/s | TTFT |
| ---: | ---: | ---: | ---: |
| 4,096 | **782.2** | **68.02** | **5.273 s** |
| 32,768 | **1,549.4** | **73.39** | **21.224 s** |
| 128,000 | **1,513.8** | **68.54** | **84.751 s** |

## Notes

- Both are optimized local configurations rather than Strata setup defaults.
- Both use a 262,144-token context limit, INT8 KV, resident KV, and MTP speculative decoding.
- No vision projector was loaded for either accepted run.
- The Q2 arm is an abliterated derivative with a workload-trained expert profile; the benchmark reports throughput, not a quality comparison.
- The 128K test does not fill the full 262K context window.
- The accepted benchmark runs were single-client, serial, and uncontaminated.

Full min/max ranges, total timings, exact measured settings, limitations, and hardware/software details are in the submitted `README.md`, `results.json`, and `summary.json`.

本站相关内容

相关页面的快捷入口。