#6535 [Benchmark] RTX 5060 Ti / Qwen3.8-Flash-Next / IQ3_S / strata-qwen38-thinking-v1-draft.1
open · @foghorn · 0 comments · View on GitHub
BenchmarksServer & APINVIDIA / CUDAModels & quantsDocumentation
Description
## Problem and result A standard-benchmark run for a card the results folder has no standard run for: one **NVIDIA GeForce RTX 5060 Ti 16 GB** with **Qwen3.8-Flash-Next IQ3_S**, protocol **`strata-qwen38-thinking-v1-draft.1`** (draft), **custom** track. Status: **complete**. Report: [`docs/bench-results/2026-11-22-standard-rtx-5060-ti-31ebb369/README.md`](https://github.com/foghorn/Strata/blob/82aaea00418a90e0bb03f26a06c5faf1a39a6c0e/docs/bench-results/2026-11-22-standard-rtx-5060-ti-31ebb369/README.md). | Case | Prompt tok/s | Decode tok/s | TTFT s | Capped | | --- | --- | --- | --- | --- | | short | not shown (fixed request cost) | 47.4 (45.0-51.9) | 1.07 | 1/3 | | prompt-4k | 816.7 (815.2-817.2) | 44.9 (43.4-46.6) | 5.05 | 3/3 | | prompt-16k | 1190.5 (1184.8-1194.7) | 45.8 (44.1-47.8) | 13.80 | 3/3 | Median (min-max), n=3. Sanity checks: **12 of 12** (instruction 4/4, retrieval 4/4, tool call 4/4). Results only: no engine, Terminal or docs changes. ## Related discussion none ## Approach Run `20261122T021914Z-31ebb369`, written by `strata bench run`. The runner started the engine itself, so `engine.log`, `config.json` and the model file hashes all come from the same launch. No file in the folder was edited by hand. - Hardware: RTX 5060 Ti 16 GB (180 W limit), Intel Core i7-10700K, 62.7 GiB RAM. Fedora 44, kernel 7.3.4, driver 620.70.06, CUDA 13.5. - Software: Strata 0.2.18 release, engine 0.1.64, commit `6e38f588a646c0ee6948a74afbe670eb75896106`. - Configuration (custom track): the `iq3_s-128k-vis` launch preset with max-context raised 131072 → 262144; the experimental speed-projection control vector (scale 1.0, layers 4-44, project/per-layer); DFlash2 speculative drafting with the selq4 pack (spec 6, draft-conf 0.7); KV int8, kv-resident 32768; vision helper loaded. One slot. - Sampling: temperature 1.0, top_p 0.95, top_k 20, reasoning effort xhigh, thinking on. Stages: the required stages (warm-up, 9 performance requests, 12 sanity checks) ran. The optional concurrency stage was not selected. ## Validation **Passed:** - `bench_standard.validate()` on `results.json` (from this checkout at `6e38f588a`): no problems. - SHA-256 of all 6 listed artifacts match `results.json`. Corpus SHA-256 is `e75eff8b…`, as the protocol requires. - The engine options in `config.json` match the "engine started:" line in `engine.log` (the only extra is the launcher's `--serve`). The draft pack hashed in `model.files` is the one the engine loaded (`qwen3.8-flash-next-dflash2.q4_0-selq4.pack`, `4a42b982…`). - `README.md` is byte-identical to `report_markdown()` regenerated from `results.json`. - Redaction: searched every file for home folders, user and host names, private addresses and keys; only the runner's `<user>` placeholders remain. **Failed:** - No request failed. 7 of 9 performance requests and the warm-up reached the 512-token cap (reasoning included), so `first_answer_s` is missing for 4 performance requests and those are left out of the first-answer summaries. **Not run:** - Optional concurrency stage. - `strata bench report` through the CLI on the reviewing machine (its Python lacked the server packages); the same validator was called directly instead. ## Limits - Draft protocol: `model_card_revision` is not pinned yet. - Not measured: model, source and tokenizer revisions; build options; storage of the model files. Reasons are in `missing` in `results.json`. - Custom track, with an experimental control vector: not comparable to default-track runs. - The control vector file is not in `model.files` and `configuration.experimental_options` is empty: the runner does not hash a file named with a `:scale` suffix, and puts the control vector options under `extensions.other_engine_options`. `--control-vector-layer-range 4 44` is recorded as `"4"` (the log shows layers 4..44). These are runner limits, to be reported separately. - Background work: the strata-vision helper process. The expert cache stays warm between requests; these are not cold-model measurements. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Related on this site
Quick links to related pages.