贡献 / #2184
#2184 bench: a 15-question checkable quality suite for head-to-head comparisons
open · @dvasdekis · 0 评论 · 去 GitHub 看
说明
### Summary Adds ench/quality_suite.py, a checkable evaluation suite designed for head-to-head model and quantization comparisons on the same box: - 15 objective questions with verifiable right answers across arithmetic, factual recall, Python synthesis, bug diagnosis, and strict instruction following. - Deliberately excludes subjective or aesthetic questions requiring manual grading. - Measures pass rate alongside warm steady-state decode speed. - Supports configurable --port, --url, and run label. ### What it leaves alone - Inference engine, server, and existing benchmarks are untouched. Zero dependencies outside standard library.
本站相关内容
相关页面的快捷入口。