#3449 [Benchmark] RTX 3090 / Qwen3.8-Flash-Next UD-Q4_K_XL / strata-qwen38-thinking-v1-draft.1

Beschreibung

## Run 1: tuned configuration (track custom)

A draft-protocol run of `strata bench run --suite standard-v1` on one RTX 3090, engine 0.1.46.1 (Strata 0.2.1 source install, commit `9ae93f7b`), with Unsloth's **UD-Q4_K_XL** GGUF read in place.

Report: [`bench/results/2026-10-20-standard-rtx-3090-b66d04c6/README.md`](bench/results/2026-10-20-standard-rtx-3090-b66d04c6/README.md)
Machine-readable result: `results.json` in the same folder, plus `outputs.jsonl` (every request, raw usage, timings, client event offsets), `config.json`, `engine.log`, and `provenance/` (chat template, corpus build, corpus manifest).

Protocol `strata-qwen38-thinking-v1-draft.1`, status **complete**, draft (not activated).

## Hardware and software

- GPU: NVIDIA GeForce RTX 3090, 24 GiB, **power limit 200 W** (`nvidia-power-limit.service`)
- CPU: Intel Core i9-9960X; 123 GiB RAM; Ubuntu 26.04.1 (kernel 7.0.0-38), driver 595.99.02, CUDA 13.2
- Storage: Samsung MZVL2512HCJQ NVMe
- Strata commit `9ae93f7b`, engine 0.1.46.1, source build, GCC 15.2, sm_86

## Model and configuration

- Model: Qwen3.8-Flash-Next, Unsloth **UD-Q4_K_XL** (dynamic quant), 4 GGUF shards read in place (no `experts.bin`); file SHA-256 prefixes are in the report.
- Context 1,048,576; KV int8 with 32,768 resident cells; MTP draft layer; `--prefill auto --spec 4 --spec-min-p 0.5`; `--resident-budget-gib 80` so all 59.4 GiB of experts stay in RAM (the run log shows 0.0 GiB read from the files); control vector (experimental speed projection) on.
- **Track: `custom`.** This is not a setup-defaults configuration: the config is hand-written, the control vector is an experimental option, and the GPU power is capped at 200 W. Cannot be merged with a `default` headline.
- One slot; the run used loopback HTTP and the server was bound to 127.0.0.1 so no other client could interfere.

## Results (medians, min-max, n=3)

| Case | Prompt tok/s | Decode tok/s | TTFT s | Total s |
| --- | --- | --- | --- | --- |
| short | 85.1 (79.0-91.2) | 58.2 (51.2-60.8) | 0.93 (0.87-1.00) | 9.1 (8.0-11.0) |
| prompt-4k | 448.8 (448.1-448.9) | 56.7 (52.8-58.3) | 9.16 (9.15-9.17) | 18.2 (17.9-18.8) |
| prompt-16k | 782.9 (773.5-786.7) | 54.8 (52.8-57.2) | 20.97 (20.87-21.23) | 30.3 (29.8-30.9) |

- Required requests: **21 of 21 completed, 0 failed, 0 skipped**. Time after the model loaded: **4 min 26 s** (faster than the 20-30 min the protocol expects; 7 of the 9 performance requests reached the 512-token cap and were truncated, which the protocol allows).
- Sanity checks: **12 of 12 passed** (instruction 4/4, retrieval 4/4, tool calls 4/4).
- No optional concurrency stage in this run.

## Validation

`strata bench report bench/results/2026-10-20-standard-rtx-3090-b66d04c6` re-validates the saved result without errors (corpus checked against `MANIFEST.sha256`, count identities, summaries, quality scores).

## Artifact SHA-256

```
6b44eed075a71faea680b01a6294db7b03662e7758fe3b54b244bf29c806a402  config.json
ec69acfc2798b3c6bd46e41acab3eb21a3b58ada124cb0ece880365528475599  engine.log
a7a5f75ddbe462952508a1ab6cc23ce1b5f025ab6b370d581786628cd43964e7  outputs.jsonl
5592f3ffb3f4484e0da4f7f73c22d7a6c2516b341d462a5b5df45738b6a27fee  README.md
cabcd8f0b08191c4f69e0ead67b6bcb5c94f73e9f810090037091400aed0e224  results.json
12827f24b742ea4e80cdc12dbcf9622227056b9f797252a3149263d4f9aaadce  provenance/chat_template.jinja
bc6c1f81d86863c3dc36a31f58010701d8ab05180f69f86c6978214409e43af2  provenance/corpus-build.json
e75eff8b9feea04f745520b392c4c17d97aefc1ba185e222aa6740efa8f68407  provenance/corpus-manifest.sha256
```


## Second run in this PR: same-machine baseline

A second `standard-v1` run on the same host, in [`bench/results/2026-10-20-standard-rtx-3090-ce9e57fe/`](bench/results/2026-10-20-standard-rtx-3090-ce9e57fe/README.md). **Everything is identical to the first run except two things:** the experimental speed-projection control vector is off, and the GPU runs at its stock **300 W** limit instead of 200 W. Also `custom` track, for the same reason (hand-written config).

21 of 21 required requests, 0 failed, 0 skipped; 12 of 12 sanity checks; 4 min 9 s after the model loaded.

| Case | Tuned (control vector on, 200 W) | Baseline (control vector off, 300 W) |
| --- | --- | --- |
| short | prompt 85.1, decode 58.2, TTFT 0.93 s | prompt 73.3, decode 55.3, TTFT 1.08 s |
| prompt-4k | prompt 448.8, decode 56.7, TTFT 9.16 s | prompt 450.8, decode 58.5, TTFT 9.11 s |
| prompt-16k | prompt 782.9, decode 54.8, TTFT 20.97 s | prompt 854.2, decode 56.4, TTFT 19.22 s |

Medians of three repetitions. The two runs change **two** variables at once (control vector and power limit), so this table is not an attribution: it shows the spread of two documented configurations of the same machine, not a measured effect of either switch. The 16K prompt difference (+9 percent) is the largest and would need one-variable-at-a-time pairs to attribute.

Artifact SHA-256 for this run:

```
6b44eed075a71faea680b01a6294db7b03662e7758fe3b54b244bf29c806a402  config.json
343c2a926a0c88a40a463a9d1579eb96d43ff953357da8929310cde82cbe72f2  engine.log
bc9c38cdf4d6a686bb74a2ce0868594ded40444bb8aaa10f036c1cb16b15d7c3  outputs.jsonl
620ebd5d85341c02ce3ca18219a83b87786e8b559da224e7e1e4faa8819eaf38  README.md
2453b788c93db016c91f60175a33f2cf18c121fb10895b9bb34fef13b003728a  results.json
12827f24b742ea4e80cdc12dbcf9622227056b9f797252a3149263d4f9aaadce  provenance/chat_template.jinja
bc6c1f81d86863c3dc36a31f58010701d8ab05180f69f86c6978214409e43af2  provenance/corpus-build.json
e75eff8b9feea04f745520b392c4c17d97aefc1ba185e222aa6740efa8f68407  provenance/corpus-manifest.sha256
```

## Limitations (also stated in the reports)

- The cache was not emptied between requests and the expert cache stays warm and evolves; these are not cold-model measurements.
- The runner cannot confirm the server was freshly loaded.
- The quantization repository revision was not measured (`null` with a reason in `results.json`).
- Prompt-prefix reuse is the shared template prefix only.
- Residency and speeds are specific to this host: 123 GiB RAM holding all experts, 200 W cap, control vector on. A 24 GB card with less RAM, or without the resident budget, will differ.
- No engine changes in this PR; results only.