#5431 [Benchmark] RTX 4070 Ti SUPER 16 GB / Qwen3.8-Flash-Next IQ2_XS / 0.2.11 engine 0.1.57

closed · @EternalShade3D · 0 コメント · GitHub で見る

BenchmarksSetup & installMulti-GPUNVIDIA / CUDAModels & quantsSecurityDocumentationWindowsLinux

本文

## Problem and result

Upstream has no standard-v1 result on 0.2.11 yet. The only committed runs under protocol `strata-qwen38-thinking-v1-draft.1` are the 0.2.1 pilot in `bench/results/2026-10-20-0.2.1-bench` (RTX 3060 and 2x A4000), and the RTX 4070 Ti SUPER 16 GiB (sm_89, Ada, single card) has exactly one committed entry, the Linux community run `2026-10-05-community-rtx-4070-ti-super` by lfontanez (upstream-main `6f32ec07`, wall-clock cold prefill plus decode at 32k-256k), which uses a different method and is not comparable row for row. This adds a standard-protocol run for the card, end to end under Windows, on the 0.2.11 release (`c3b3f503`) with engine 0.1.57.

Run `20261106T200829Z-dc2b2083`, protocol `strata-qwen38-thinking-v1-draft.1`, status **complete**: 21 of 21 required requests finished, 0 failed, 0 skipped, **12 of 12** sanity checks, 1 min 44 s after the model loaded. Report: `bench/results/2026-11-06-standard-rtx-4070-ti-super-dc2b2083/README.md`.

| Case | Prompt tok/s | Decode tok/s | TTFT s | First answer s | Total s | Capped |
| --- | --- | --- | --- | --- | --- | --- |
| short | n/a (about 80 tokens) | 102.6 (95.4-106.3) n=3 | 0.50 (0.41-0.57) n=3 | 2.48 (2.37-2.73) n=3 | 4.5 (4.0-4.6) n=3 | 0/3 |
| prompt-4k | 1871.3 (1863.3-1880.6) n=3 | 97.0 (92.7-107.8) n=3 | 2.22 (2.20-2.24) n=3 | 5.74 (4.62-6.86) n=2 | 7.5 (6.9-7.7) n=3 | 3/3 |
| prompt-16k | 2396.6 (2375.5-2500.0) n=3 | 106.1 (90.4-109.7) n=3 | 6.88 (6.61-6.93) n=3 | 8.61 (7.88-9.33) n=2 | 11.7 (11.5-12.3) n=3 | 3/3 |

Sanity **12 of 12** (instruction 4/4, retrieval 4/4, tool calls 4/4). Six of the nine performance requests reached the 512-token cap while still reasoning, so two `first_answer_s` values are `null` and are listed as excluded in the report. The short case has no prompt rate on purpose: a prompt of about 80 tokens is dominated by the fixed cost of a request, so prompt speed is read from the 4K and 16K rows.

## Related discussion

None. Results only: no engine, code or documentation change.

The same machine and configuration were reported in #4716 on 0.2.10 / engine 0.1.56; that PR was closed and its numbers were collected for the benchmarks page. This folder replaces them with a current-release run, and is a new run, not a rewrite of the old one.

## Approach

One folder as `strata bench run` wrote it: `README.md`, `results.json`, `outputs.jsonl`, `config.json`, `engine.log` and `provenance/` (`chat_template.jinja`, `corpus-build.json`, `corpus-manifest.sha256`). No model files, packs or generated artifacts. `engine.log` is present because the runner started the engine itself for this run.

Hardware and software:

- GPU: NVIDIA GeForce RTX 4070 Ti SUPER, 16.0 GiB dedicated, 300 W; Ryzen 9 9900X; 61.6 GiB RAM; WD_BLACK SN850X NVMe
- OS Windows 10.0.26200; gpu-driver `616.92`; cuda `13.0` (driver supports 13.4)
- Strata `c3b3f5035506f3e966f1b4b7b188aeaabae92f1d` (v0.2.11); engine `0.1.57` (prebuilt package `0.1.57-windows-x86_64-cuda-cuda13`); track `custom`
- Model `qwen3.8-flash-next-iq2_xs`, quantization `IQ2_XS`, revision `d4af3be5701549435e0770d625f79dd2def1358f`, source revision `de4b8e4d43b917e7706784d8bb445c9af86a3540`
- Weights `alesha-pro/Qwen3.8-Flash-Next-abliterated-GSQ-RCO-Strata-GGUF`; upstream `Qwen/Qwen3.8-Flash-Next`; files `Qwen3.8-Flash-Next-GSQ-RCO-IQ2_XS-00001-of-00002.gguf` `92cee27ae5bbadcd...` and `...-00002-of-00002.gguf` `316b46f3a2dbd68c...`
- `--custom-pack` `expert-profile.bin` sha256 `8f59b4aa8873209dff11c11e37bcda9529a1335b724a1afeea37bf6388975baf`; refusal-direction control vector `Huihui-Qwen3.8-Flash-Next-refusal-direction-r.gguf` sha256 `59f9ae8b4372107ae79044278a4406bdfe8f8861fe599aa5848515f0d072be38`, scale 1.0, layers 4-44, `--cvec-mode project`, `--cvec-dir per-layer`; MTP draft pack `mtp/rt`
- `--kv int8`, `--kv-resident 32768`, `--expert-cache auto`, `--prefill auto`, `--spec 4 --spec-min-p 0.5`, `--vram-reserve-mib 700`, `--conversation-cache-mib 8192 --conversation-cache-slots 4 --conversation-cache-min-tokens 2048`, `reasoning_effort` `xhigh`

## Validation

**Passed:** the run itself, command exactly as issued:

```
strata bench run --profile strata-iq2_xs.json --track custom --track-notes "0.2.11 source install, engine 0.1.57 (prebuilt cuda13), abliterated IQ2_XS (alesha-pro GSQ-RCO), refusal control vector layers 4-44, 128K context, 1 slot" --model-revision d4af3be5701549435e0770d625f79dd2def1358f --source-revision de4b8e4d43b917e7706784d8bb445c9af86a3540 --quantization IQ2_XS --custom-pack "expert-profile.bin sha256:8f59b4aa...; refusal-direction control vector Huihui-Qwen3.8-Flash-Next-refusal-direction-r.gguf sha256:59f9ae8b..., scale 1.0, layers 4-44, cvec-mode project, cvec-dir per-layer; MTP draft pack mtp/rt" --background "desktop idle, no other GPU work"
```

at commit `c3b3f503`: status `complete`, `request_counts` warmup 1/1 attempted (truncated at the cap), performance 9/9 (3 completed, 6 truncated, 0 failed), sanity 12/12, concurrency 0 (not selected); `run.required_duration_s` 104.641 s; `server_requests_before_run` 0 and `started_engine` true (the runner started the engine, hence `engine.log`); exit 0.

**Passed:** `strata.py bench report bench/results/2026-11-06-standard-rtx-4070-ti-super-dc2b2083` re-validates the folder and prints status `complete`, 21 of 21 required requests, 0 failed, 0 skipped, sanity 12/12; exit 0.

**Passed:** artifact hashes. Every entry of `results.json` `artifacts[]` re-hashed **from the committed git blobs**, not from the working copy, with `hashlib.sha256`: `outputs.jsonl`, `config.json`, `engine.log`, `provenance/chat_template.jinja`, `provenance/corpus-manifest.sha256`, `provenance/corpus-build.json` - 6 of 6 match, so `core.autocrlf` did not alter any of them on the way into the commit. `sha256sum` of the two main files: `results.json` `7fc7a5f211f31f21...`, `outputs.jsonl` `3ce3d5c705af446d...`.

**Passed:** privacy scan. `grep -rinE 'Eternal|C:\Users\[A-Za-z]|192\.168\.'` over the folder, after excluding the runner's own `<user>` / `<strata>` placeholders: 0 hits. No user name, real home path, LAN address, host name or credential value in any added file.

**Failed:** nothing observed.

**Not run:** the optional four-client concurrency stage (the server has one slot); a source build, because the prebuilt `0.1.57-windows-x86_64-cuda-cuda13` package was used, so `software.build_options` is `null` with the runner's own reason; `--tokenizer-revision` not passed, because the tokenizer is the one inside the IQ2_XS GGUF files and no separate revision was recorded, and no revision was invented for it.

## Limits

One configuration, one run, one card. The protocol flags a second run against the same engine as `nonconforming`, so no same-machine repeat is included inside this folder. The expert cache is warm and evolves during the run, and prompt-prefix reuse is the shared template prefix only, so these are not cold-model measurements. Six of nine performance requests reached the 512-token cap while still reasoning: their totals are time-to-cap, not time-to-answer. Clocks were not sampled per request. The numbers describe a prebuilt CUDA 13.0 engine on an abliterated IQ2_XS derivative with a refusal-direction control vector, so they are not directly comparable with an official release running the original Qwen weights.

Two things a reviewer may want to know: the `<redacted>` value in `config.json` is a single key, `--conversation-cache-min-tokens` (shown in both the engine options and the server defaults), which the runner redacts because it carries a token count; nothing else is withheld. `protocol.model_card_revision`, `model.tokenizer_revision` and `software.build_options` are `null` for the reasons given above, while `run.optional_duration_s` is `null` because the concurrency stage was not selected.

関連リンク

インストール・モデル・リリースへのサイト内リンク。