#3535 [Benchmark] RTX 2080 Ti x2 / Qwen3.8-Flash-Next / GSQ-RCO IQ3_XXS / strata-qwen38-thinking-v1-draft.1

说明

## Problem and result

The 0.2.1 standard benchmark protocol is still a draft, and it needs pilot data from more than one kind of hardware. This PR adds three complete standard-benchmark runs for a two-card Turing setup (2x RTX 2080 Ti, compute capability 7.5, PCIe 3.0 x8) with the GSQ-RCO IQ3_XXS quantization of Qwen3.8-Flash-Next.

Result: in each run, 21 of 21 required requests completed, 0 failed, 0 skipped; prompt speed agrees within about 1% across the three runs; the sanity score is **not** stable for this configuration (11/12, 11/12, 12/12) because the tool-call case `sanity-T2` sits on a borderline token branch. All three runs are submitted; the best one alone is not.

The protocol's draft status is visible in every report: `strata-qwen38-thinking-v1-draft.1`.

### Reports

| Run | Folder | Sanity | Duration after load |
|---|---|---:|---:|
| 1 | [`2026-10-21-standard-rtx-2080-ti-plus-rtx-2080-ti-0dd986c2`](bench/results/2026-10-21-standard-rtx-2080-ti-plus-rtx-2080-ti-0dd986c2) | 11/12 | 2 min 43 s |
| 2 | [`...-124a977a`](bench/results/2026-10-21-standard-rtx-2080-ti-plus-rtx-2080-ti-124a977a) | 11/12 | 2 min 40 s |
| 3 | [`...-09a9bf8e`](bench/results/2026-10-21-standard-rtx-2080-ti-plus-rtx-2080-ti-09a9bf8e) | 12/12 | 2 min 51 s |

### Machine

- 2x NVIDIA GeForce RTX 2080 Ti (22.0 GiB each), compute capability 7.5, driver 595.99.02
- Intel Core i7-10700K (AVX2, no AVX-512), 94.1 GiB RAM, Ubuntu 24.04.5
- **PCIe 3.0 x8** (measured 6.5 GB/s host->device), NVMe 970 Pro
- All 39.97 GiB of experts resident in RAM, ~14 GiB expert cache in VRAM

### Model and configuration

- `qwen3.8-flash-next-iq3_xxs`, quantization **GSQ-RCO IQ3_XXS** (`ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF`, revision `ed59f92082b1e93c0e96d60a8b11aab089b52f09`)
- Source weights / tokenizer revision `de4b8e4d43b917e7706784d8bb445c9af86a3540` (Qwen/Qwen3.8-Flash-Next)
- Context 262144, 3 slots, KV int8, `--layer-split 26`, `--vram-reserve-mib 1500`, `--spec 4 --spec-min-p 0.5`, MTP on
- Engine environment: `STRATA_SM75_PERF=1`, `STRATA_BF16_TC=1`, `STRATA_PA_V2T=1` (the Turing paths documented in docs/OLDER_GPUS.md); `--batch 3` comes from the server's 3 slots
- The engine was compiled locally for sm_75 with CUDA 12.8; the ready-made package was not used

### Speed (median of the three repetitions in each run)

| Case | Prompt tok/s | Decode tok/s | TTFT s |
|---|---|---|---|
| short | 131.6 - 133.7 | 76.6 - 85.2 | 0.60 - 0.61 |
| prompt-4k | 919.8 - 922.1 | 69.1 - 76.4 | 4.46 - 4.47 |
| prompt-16k | 1590.4 - 1593.0 | 68.0 - 73.5 | 10.32 - 10.34 |

## Related discussion

None for this submission. The same machine's earlier, non-standard tuning results (the sm_75 engine switches, measured with the pre-protocol tool) are in #3282.

## Approach

Results only: three folders under `bench/results/`, one per run, and no change to the engine, the tools or the protocol. Each folder holds the runner's own `README.md`, `results.json`, `outputs.jsonl` (every request with its usage, engine timings and client event offsets), `config.json` and `provenance/` (corpus manifest and the chat template used).

One extra file is added per folder, `engine-requests.log`: the runner's `engine.log` is its capture of the server's stdout, which is empty because the runner starts the server with `--quiet`, so the engine's own request log is included instead. It is redacted with the runner's own `redact()` helper, as are all other artifacts.

Nothing else was changed in the repository.

## Validation

**Passed:**

- `strata bench run --profile <config> --suite standard-v1 --track default --model-revision ed59f92082b1e93c0e96d60a8b11aab089b52f09 --source-revision de4b8e4d43b917e7706784d8bb445c9af86a3540 --tokenizer-revision de4b8e4d43b917e7706784d8bb445c9af86a3540 --quantization "GSQ-RCO IQ3_XXS"` - three runs on Strata commit `9de04cd44e927a9a3df6bbc0e94fc1be8b844ba4` (v0.2.1.4), engine 0.1.46.3 built from source for sm_75, Ubuntu 24.04.5, driver 595.99.02 - every run reports `Status: complete. 21 of 21 required requests finished, 0 failed, 0 skipped`, and `strata bench report <folder>` re-validates each saved result.
- The corpus check: the runner verified all 34 frozen fixtures against the bundle manifest before sending anything and refused to run on a single differing byte.
- Redaction: the user name, the private addresses and the API key do not appear in any artifact of the three folders (0 occurrences each).

**Failed:**

- `sanity-T2` ("Use get_weather for Zagreb in celsius") in runs 1 and 2: the model narrates the call instead of emitting it. The failing output is a long thinking trace of 512 - 1211 output tokens ending in a sentence like "I requested the weather for Zagreb in Celsius using `get_weather`", with `tool_calls: []`; a successful run of the same request uses 63 - 94 output tokens and returns a well-formed call. Repeating the identical request in isolation succeeded 17 of 19 times, and the two failures were the first request on a freshly started engine. The model's first token on this request is on a borderline branch, so small numeric differences (cold expert cache, batch state) decide whether it calls the tool or describes the call. It is not a template or parser problem: the same server returns a well-formed `tool_calls` entry for the same request, and `sanity-T1` (a one-parameter tool) passed in all three runs.

**Not run:**

- The optional concurrency stage of the protocol (`--optional-concurrency`): not required by the protocol, so it was not run.
- The same protocol on other quantizations, other GPUs and other backends (AMD/HIP, SYCL, Metal): this machine has two NVIDIA cards, so nothing else was measured. In particular, a run with the abliterated IQ3_S weights (a different model, not this one) is not part of this PR.
- Storage speed of the model files and background work on the machine: both are reported as "not measured" in the JSON.

## Limits

- This is pilot data for a draft protocol. It is not an approved result and does not claim to be one.
- The speed numbers describe a two-card Turing setup on PCIe 3.0 x8 and are not comparable with the PCIe 4.0+ cards in the existing community reports.
- The sanity score for this configuration is unstable (11/12, 11/12, 12/12). A single run of this configuration cannot be read as "12/12" or as "11/12"; that is why all three runs are here.
- The decode numbers have a wider spread than the prompt numbers (76.6 - 85.2 tok/s on the short case across runs), which is the run-to-run spread of the server, not a difference between the runs' settings.
- `engine.log` in each folder is empty by construction (the runner's capture of the server's stdout under `--quiet`); `engine-requests.log` holds the engine's own request log for that run.