#3549 [Benchmark] RTX 2080 Ti x2 / Qwen3.8-Flash-Next GSQ-RCO abliterated (SC117) IQ3_S / strata-qwen38-thinking-v1-draft.1

说明

## Problem and result

The standard benchmark protocol is a draft and needs pilot data from more than one kind of hardware. This is a second standard-protocol run of the **same** box with a **different pack**: the abliterated IQ3_S (SC117) instead of the IQ3_XXS of PR #3535. It shows the protocol end to end on a 3.5-bit i-quant pack, on a derivative model, and it is the only one of our four runs that passed every sanity check.

Result: **21 of 21 required requests completed, 0 failed, 0 skipped, sanity 12/12** (instruction 4/4, retrieval 4/4, tool calls 4/4), 2 min 54 s after the model had loaded.

### Report

[`bench/results/2026-10-21-standard-rtx-2080-ti-plus-rtx-2080-ti-12f18379`](bench/results/2026-10-21-standard-rtx-2080-ti-plus-rtx-2080-ti-12f18379) - protocol `strata-qwen38-thinking-v1-draft.1` (draft), run `20261021T021121Z-12f18379`, status `complete`.

### Machine

- 2x NVIDIA GeForce RTX 2080 Ti (22.0 GiB each), compute capability 7.5, driver 595.99.02, NVLink NV2 (2 links, 25.78 GB/s)
- Intel Core i7-10700K (AVX2, no AVX-512), 94.1 GiB RAM, Ubuntu 24.04.5
- **PCIe 3.0 x8** (measured 6.5 GB/s host->device), NVMe 970 Pro
- All 47.8 GiB of experts resident in RAM

### Model and configuration

- `qwen3.8-flash-next-iq3_s-abl`, quantization **GSQ-RCO IQ3_S, abliterated** (`SC117/Qwen3.8-Flash-Next-GSQ-RCO-abliterated-GGUF`, revision `eec4e10a2c29b440a2ae85591fa11569ce752963`)
- Source weights / tokenizer revision `de4b8e4d43b917e7706784d8bb445c9af86a3540` (Qwen/Qwen3.8-Flash-Next) - the abliteration is applied to a derivative of these weights, so the source revision points at the base model
- Context 262144, 3 slots, KV int8, `--layer-split 26`, `--vram-reserve-mib 1500`, `--spec 4 --spec-min-p 0.5`, MTP on
- Engine environment: `STRATA_SM75_PERF=1`, `STRATA_BF16_TC=1`, `STRATA_PA_V2T=1`; `--batch 3` comes from the server's 3 slots
- The engine was compiled locally for sm_75 with CUDA 12.8; the ready-made package was not used
- Strata commit `9de04cd44e927a9a3df6bbc0e94fc1be8b844ba4` (v0.2.1.4), engine **0.1.46.3**

### Speed (median of three repetitions)

| Case | Prompt tok/s | Decode tok/s | TTFT s |
|---|---|---|---|
| short | 86.4 | 69.8 | 0.91 |
| prompt-4k | 746.4 | 71.5 | 5.51 |
| prompt-16k | 1488.7 | 68.3 | 11.04 |

The IQ3_S pack is 3.5-bit against the IQ3_XXS pack's 3-bit, and it shows in the prompt path: 746 tok/s at 4K against 922 tok/s for IQ3_XXS on the same box, with the decode rate close (69.8 against 76.6 tok/s).

## Related discussion

- #3282 - this machine's earlier, non-standard community report
- PR #3535 - the three IQ3_XXS runs of the same protocol on the same box
- #2197 - the sm_75 kernel series and expert mode; the engine environment above is the one that thread is about

## Approach

Results only: one folder under `bench/results/`, no change to the engine, the tools or the protocol. The folder holds the runner's own `README.md`, `results.json`, `outputs.jsonl` (every request with its usage, engine timings and client event offsets), `config.json` and `provenance/` (corpus manifest and the chat template used).

One extra file is added, `engine-requests.log`: the runner's `engine.log` is its capture of the server's stdout, which is empty because the runner starts the server with `--quiet`, so the engine's own request log is included instead. It is redacted with the runner's own `redact()` helper, as are all other artifacts.

## Validation

**Passed:**

- `strata bench run --profile <config> --suite standard-v1 --track custom --model-revision eec4e10a2c29b440a2ae85591fa11569ce752963 --source-revision de4b8e4d43b917e7706784d8bb445c9af86a3540 --tokenizer-revision de4b8e4d43b917e7706784d8bb445c9af86a3540 --quantization "GSQ-RCO IQ3_S (SC117 abliterated)"` - one run on Strata commit `9de04cd44e927a9a3df6bbc0e94fc1be8b844ba4` (v0.2.1.4), engine 0.1.46.3 built from source for sm_75, Ubuntu 24.04.5, driver 595.99.02 - `Status: complete. 21 of 21 required requests finished, 0 failed, 0 skipped`, and `strata bench report <folder>` re-validates the saved result.
- The corpus check: the runner verified all 34 frozen fixtures against the bundle manifest before sending anything and refuses to run on a single differing byte.
- Redaction: the user name, the private addresses and the API key do not appear in any artifact (0 occurrences each).

**Failed:**

- None. All twelve sanity cases passed, including the tool-call case, which is where the IQ3_XXS runs of PR #3535 were not stable.

**Not run:**

- The optional concurrency stage of the protocol (`--optional-concurrency`): not required by the protocol.
- The same protocol on other GPUs, other backends or the non-abliterated IQ3_S weights: this machine has two NVIDIA cards and one model, so nothing else was measured. The original-weights IQ3_S profile is not part of this PR.
- Storage speed of the model files and background work on the machine: both are reported as "not measured" in the JSON.

## Limits

- **This is a derivative model, not the original Qwen/Qwen3.8-Flash-Next weights.** The protocol asks for a separate profile for derivatives, so this run is labelled `--track custom` with that stated in the track notes, and its numbers are not comparable with a run of the original weights. It is here as a second pack on the same hardware, not as a substitute for the canonical profile.
- This is pilot data for a draft protocol, not an approved result.
- The speed numbers describe a two-card Turing setup on PCIe 3.0 x8 and are not comparable with the PCIe 4.0+ cards in the existing community reports.
- The decode numbers have a wider spread than the prompt numbers (68.4-73.4 tok/s on the short case), which is the run-to-run spread of the server.
- `engine.log` is empty by construction (the runner's capture of the server's stdout under `--quiet`); `engine-requests.log` holds the engine's own request log for this run.