贡献 / #2332

#2332 Add community benchmarks: RTX 4090 Laptop, IQ3_XXS and IQ3_S, default vs calibrated

open · @delirious6423 · 0 评论 · 去 GitHub 看

BenchmarksNVIDIA / CUDAModels & quantsLinux

说明

Results-only submission, no engine changes. Five single `tools/strata_bench.py --long` runs on one machine (RTX 4090 Laptop 16 GB, i7-13700HX, 61 GB RAM, NVMe SSD, Linux, CUDA 13.1 build), Qwen3.8-Flash-Next, 64K context.

| Report | Engine | Settings | Short-chat decode tok/s | Agent decode tok/s | 32K prompt tok/s |
| --- | --- | --- | ---: | ---: | ---: |
| IQ3_XXS default | 0.1.43.1 | defaults | 46.0 | 63.3 | 1731 |
| IQ3_XXS calibrated | 0.1.43.1 | `--calibrate` | 58.5 | 70.8 | 1746 |
| IQ3_S default | 0.1.44 | defaults | 44.0 | 52.3 | 1403 |
| IQ3_S calibrated | 0.1.44 | `--calibrate` | 48.2 | 57.1 | 1442 |
| IQ3_S calibrated, `performance` power profile | 0.1.44 | `--calibrate` | 56.9 | 68.2 | 1510 |

Calibration mostly raised decode speed (+27% IQ3_XXS, +9.5% IQ3_S short chat) and left prompt reading about unchanged. Each report lists its kept settings.

Limitations: one suite run per configuration, so differences of a few percent are within run-to-run variation. The IQ3_XXS and IQ3_S rows are on different engine versions. The power profile of the earlier runs was not recorded; background load and GPU power limits were not recorded. Speed only, not a quality test.

本站相关内容

相关页面的快捷入口。