#6535 [Benchmark] RTX 5060 Ti / Qwen3.8-Flash-Next / IQ3_S / strata-qwen38-thinking-v1-draft.1

open · @foghorn · 0 commentaires · Sur GitHub

BenchmarksServer & APINVIDIA / CUDAModels & quantsDocumentation

Description

## Problem and result

A standard-benchmark run for a card the results folder has no standard run for: one **NVIDIA GeForce RTX 5060 Ti 16 GB** with **Qwen3.8-Flash-Next IQ3_S**, protocol **`strata-qwen38-thinking-v1-draft.1`** (draft), **custom** track. Status: **complete**. Report: [`docs/bench-results/2026-11-22-standard-rtx-5060-ti-31ebb369/README.md`](https://github.com/foghorn/Strata/blob/82aaea00418a90e0bb03f26a06c5faf1a39a6c0e/docs/bench-results/2026-11-22-standard-rtx-5060-ti-31ebb369/README.md).

| Case | Prompt tok/s | Decode tok/s | TTFT s | Capped |
| --- | --- | --- | --- | --- |
| short | not shown (fixed request cost) | 47.4 (45.0-51.9) | 1.07 | 1/3 |
| prompt-4k | 816.7 (815.2-817.2) | 44.9 (43.4-46.6) | 5.05 | 3/3 |
| prompt-16k | 1190.5 (1184.8-1194.7) | 45.8 (44.1-47.8) | 13.80 | 3/3 |

Median (min-max), n=3. Sanity checks: **12 of 12** (instruction 4/4, retrieval 4/4, tool call 4/4).

Results only: no engine, Terminal or docs changes.

## Related discussion

none

## Approach

Run `20261122T021914Z-31ebb369`, written by `strata bench run`. The runner started the engine itself, so `engine.log`, `config.json` and the model file hashes all come from the same launch. No file in the folder was edited by hand.

- Hardware: RTX 5060 Ti 16 GB (180 W limit), Intel Core i7-10700K, 62.7 GiB RAM. Fedora 44, kernel 7.3.4, driver 620.70.06, CUDA 13.5.
- Software: Strata 0.2.18 release, engine 0.1.64, commit `6e38f588a646c0ee6948a74afbe670eb75896106`.
- Configuration (custom track): the `iq3_s-128k-vis` launch preset with max-context raised 131072 → 262144; the experimental speed-projection control vector (scale 1.0, layers 4-44, project/per-layer); DFlash2 speculative drafting with the selq4 pack (spec 6, draft-conf 0.7); KV int8, kv-resident 32768; vision helper loaded. One slot.
- Sampling: temperature 1.0, top_p 0.95, top_k 20, reasoning effort xhigh, thinking on.

Stages: the required stages (warm-up, 9 performance requests, 12 sanity checks) ran. The optional concurrency stage was not selected.

## Validation

**Passed:**
- `bench_standard.validate()` on `results.json` (from this checkout at `6e38f588a`): no problems.
- SHA-256 of all 6 listed artifacts match `results.json`. Corpus SHA-256 is `e75eff8b…`, as the protocol requires.
- The engine options in `config.json` match the "engine started:" line in `engine.log` (the only extra is the launcher's `--serve`). The draft pack hashed in `model.files` is the one the engine loaded (`qwen3.8-flash-next-dflash2.q4_0-selq4.pack`, `4a42b982…`).
- `README.md` is byte-identical to `report_markdown()` regenerated from `results.json`.
- Redaction: searched every file for home folders, user and host names, private addresses and keys; only the runner's `<user>` placeholders remain.

**Failed:**
- No request failed. 7 of 9 performance requests and the warm-up reached the 512-token cap (reasoning included), so `first_answer_s` is missing for 4 performance requests and those are left out of the first-answer summaries.

**Not run:**
- Optional concurrency stage.
- `strata bench report` through the CLI on the reviewing machine (its Python lacked the server packages); the same validator was called directly instead.

## Limits

- Draft protocol: `model_card_revision` is not pinned yet.
- Not measured: model, source and tokenizer revisions; build options; storage of the model files. Reasons are in `missing` in `results.json`.
- Custom track, with an experimental control vector: not comparable to default-track runs.
- The control vector file is not in `model.files` and `configuration.experimental_options` is empty: the runner does not hash a file named with a `:scale` suffix, and puts the control vector options under `extensions.other_engine_options`. `--control-vector-layer-range 4 44` is recorded as `"4"` (the log shows layers 4..44). These are runner limits, to be reported separately.
- Background work: the strata-vision helper process. The expert cache stays warm between requests; these are not cold-model measurements.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Sur le site

Liens install, modèles, releases.