Pull requests / #3249
#3249 [Benchmark] Community benchmark: RTX 4060 Ti 16 GB, Coder IQ1_M
open · @JoShMiQueL · 0 comments · View on GitHub
BenchmarksNVIDIA / CUDAModels & quantsDocumentation
Description
## Problem and result A community report for hardware the maintainers do not have: an NVIDIA GeForce RTX 4060 Ti 16 GB running the Coder IQ1_M quant, measured 2026-10-13 on engine 0.1.43 (project 0.1.44, commit `ed4773eb`). The pull request adds `bench/results/2026-10-13-community-rtx-4060-ti-qwen3-8-flash-next-coder-iq1-m/`: the tool-written `README.md`, the per-request `results.json`, `engine-log-excerpt.txt`, the sanitized launch configuration and the engine's `BUILD.json`, so the run can be repeated. Main numbers, 3 runs each, medians with ranges, engine timings with client TTFT: prompt 1104.9 (980.0-1146.3) / 1343.9 (1334.9-1363.1) / 1265.0 (1043.4-1369.5) tok/s at 4K / 16K / 32K fresh prompts; short-chat decode 33.0 (16.0-40.0) tok/s; agent conversation cache hit 68.6 %; 0 failures. ## Related discussion Reopening of #2112, closed in the 0.2.0-beta tracker reset of 2026-10-19. The `/reopen` check passed there (area prefix in the title, template filled in, revision named), but GitHub refused the reopen itself with `state cannot be changed. The community-bench-rtx4060ti-iq1m branch was force-pushed or recreated.` — the branch was rebased onto `main` while the pull request was closed, so the bot's own advice applies: open a new pull request and link this one. This is that pull request; #2112 stays as the discussion thread. ## Approach Results only: one folder, no code and no docs, the shape `docs/COMMUNITY_BENCHMARKS.md` asks for in "Submit a report", so a reviewer can judge the measurements on their own. The index row in `bench/results/COMMUNITY.md` and the docs list are follow-ups the maintainers commit afterwards (as for #1739, #1751, #1784, #1793, #1799, #1802, #1811). ## Validation **Passed:** - The branch rebases on `main` `e88ee39d` (0.2.0-beta) with no conflicts, and the folder's five files are byte-identical to the previous revision of this pull request (`git diff` over the folder is empty). - The README read against the report template of `docs/COMMUNITY_BENCHMARKS.md`: hardware and software, model and configuration, method, results table, per-request data and limitations are present, with `not measured` where the run did not measure it. `results.json` carries `schema: 1`, the tool's `SCHEMA` on `main`. - Nothing in the folder was edited after the run: paths, user names and keys are the tool's own placeholders. **Failed:** - none observed **Not run:** - No re-measurement on 0.2.0-beta. These numbers are from 2026-10-13 on engine 0.1.43 and the report says so wherever a version is printed. The default command does not run on `main` either (`python tools/strata_bench.py --config <config> --port 9` ends in `NameError: name 'port' is not defined`, from #2330), so re-running the suite there was not done and no 0.2.0-beta numbers are claimed. - No correctness check: speed only, as the limitations section of the report states. ## Limits - Speed only: no needle, recall or coding-correctness check was run for this run. - Background load during the run was not recorded; the power figure in the report is the post-suite `nvidia-smi` sample. - The folder was written by the branch revision of 2026-10-13, so its `GPU detail` line predates the `(after the run, near idle)` wording #2111 gained on 2026-10-15 after review. Same numbers, clearer label in the tool now; the folder is left exactly as the run wrote it.
Related on this site
Quick links to related pages.