#6367 [Benchmark] unlocked RTX 4080 (sm_89) / Qwen3.8-Flash-Next (IQ3_S · Coder IQ1_M · sc117 · UD-Q4_K_XL) / strata-qwen38-thinking-v1-draft.1

open · @krabokiller · 0 comentários · No GitHub

BenchmarksServer & APINVIDIA / CUDAModels & quantsSecurityDocumentationLinux

Descrição

# unlocked RTX 4080 (sm_89, Ada AD103) on an AVX2 DDR4 desktop — 8 standard-protocol runs + long-context extras

Results-only PR (per CONTRIBUTING: no engine changes). Everything below was measured by me on this machine on
2026-11-20 (single night, same build), on a **factory-unlocked AD103** card (`gpu_unlock` DKMS: max SM 3135 MHz
vs retail 2610, power limit 340 W vs 320 W; VRAM bandwidth unchanged at 716.8 GB/s) — a configuration the
maintainers do not have, so all power/clock facts are given and the runs are labelled track=custom.

## Hardware / software

| | |
| --- | --- |
| GPU | GeForce RTX 4080 **16 376 MiB**, `sm_89`, **unlocked** (gpu_unlock/2.0.0 DKMS, kernel 7.3.5-zen1) |
| CPU / RAM | Ryzen 9 5950X (16C/32T, **AVX2 only**), 126 GB DDR4-3200 quad-channel |
| Link / disk | PCIe 4.0 x16 (engine transfer probe: 24 GB/s idle-class, 13.1 GB/s observed under desktop load), NVMe |
| OS / driver | Garuda Linux (Arch), kernel 7.3.5-zen1, NVIDIA open-kernel-module 620.73.04 |
| Strata | **v0.2.16 / engine 0.1.62 source build** (CUDA 13, archs [89], vision-cpu), commit `64524fb9f`; Control 0.1.62; server = `serve/server.py` |
| Background | KDE Plasma desktop + browser + Discord/Telegram live during every run (the #2065 free-line discipline held: `--vram-reserve-mib 1400`/`896`, **0 warnings all night**) |

## Standard runs (`strata bench run`, protocol `strata-qwen38-thinking-v1-draft.1`)

One fresh engine load per run unless noted; all eight: **sanity 12/12** (instruction/retrieval/tool 4/4 each),
0 failed / 0 skipped requests. b4–b9 status `complete`; **b1/b2 are `nonconforming` only because the COMMIT
provenance file was missing at run time** (added afterwards; no workload change). Optional concurrency stage:
not selected for any run.

| Run | Arm | 4K prompt | 16K prompt | decode (short/4K/16K) | TTFT 16K |
| --- | --- | ---: | ---: | --- | ---: |
| b1 | IQ3_S stock | 1,015.4 | 1,992.6 | 71.4 / 62.8 / 65.2 | 8.26 |
| b2 | + `STRATA_PF_FUSED=1` | 1,017.3 | 2,001.9 | 72.8 / 62.2 / 64.9 | 8.23 |
| b4 | + alesha refusal-direction vector (project, 1–47) | 1,026.9 | 2,006.1 | 69.6 / 66.6 / 65.5 | 8.20 |
| b5 | sc117-abliterated @ 1,048,576 (yarn4 k8v4 direct) | 976.0 | 1,217.1 | 66.5 / 62.9 / 60.4 | 13.51 |
| b6 | Coder IQ1_M @ 786,432 (yarn3) | 1,917.6 | 3,579.8 | 67.4 / 63.6 / 63.8 | 4.63 |
| b7 | vector + fused (prod candidate) | 1,027.9 | 2,004.4 | 73.3 / 62.8 / 65.5 | 8.21 |
| b8 | UD-Q4_K_XL @ 262,144 — DFlash2 (Auto picked it, serve-mode) | 652.5 | 1,300.5 | 34.7 / 34.9 / 34.0 | 12.67 |
| b9 | UD-Q4_K_XL @ 262,144 — MTP forced | 652.4 | 1,306.1 | 33.4 / 35.4 / 35.0 | 12.61 |

## What these numbers say (facts above; interpretation marked)

- **DFlash2 = MTP on sm_89** (b8 vs b9 within ≤1.5 % on every metric, acceptance 72.5 % vs ~70 %): the older
  CLI-era ×0.925 I measured on 0.1.51 no longer transfers to the 0.1.62 serve path. (Interpretation; both runs
  are in this PR.)
- **The UD-Q4_K_XL pack halves generation speed on a 16 GB card** (decode 33–35 vs 62–71 for IQ3_S; cold-53K
  1,429 vs ~3,050 tok/s on the same box): 3.99 MB blobs → 1,289–1,601 VRAM slots (~6.5 % coverage vs 13.7 %),
  77 GB experts served from buffered file cache while zram absorbed pressure. The all-in-RAM ×1.34–1.40 mode
  does not rescue the small-card cache geometry. Long-context extras (nonstandard, in the companion report):
  skip-parking fired at 482K (int8, 24 GiB pool), and a 43 %-position needle **failed at only 89K distance**
  where sc117/IQ3_S arms hit at 290–554K under the same k8v4/direct settings. On 16 GB VRAM, IQ3_S remains the
  right pack; 93 % top-token quality did not buy the speed/recall cost here.
- **sc117 @1M is healthy on 0.1.62** (b5: 12/12 at 1 048 576; depth probes 52.1 tok/s @500K with needle hits;
  its 16K-prefill 1,217 reproduces the 1,217–1,438 band we measured on 0.1.53–0.1.59 — not a regression).
- **Coder IQ1_M prefill ≈ ×1.8** (1,918/3,580; TTFT 4K 2.17 s) — with 49.7 tok/s decode at 714K but a far-needle
  miss at 411K distance: fast worker lane, not a deep coordinator. Also: **#5104 confirmed fixed** — the
  256-expert pack reads multi-chunk prompts fine without the `STRATA_PREFILL_LAYER_MAJOR=0` workaround.
- **`STRATA_PF_FUSED=1` on this sm_89 box**: +0.2…0.5 % on the protocol's 4K/16K (noise), **+2.0 % on our 53K
  instrument** (3,050.6 → 3,111.6 tok/s). The docs' +8.9 % (0.1.39-era, 4080 SUPER 32 GB) has largely been
  absorbed by the 0.1.60–62 own-kernel prompt path. (Interpretation.)
- The control vector (project mode) costs **nothing measurable** (b4/b7 = noise band vs b1) — consistent with
  the docs' 0.2–0.4 %-per-token note being at/below resolution on this card.

## Method / reproduction

Dedicated loopback HTTP on the inference machine, one engine per arm, freshly loaded; sampling exactly as the
protocol prescribes (xhigh thinking, temperature 1.0/top_p 0.95/top_k 20, seeds 1–3, 512 cap). Production
configs (track=custom): `--pool-workers 12 --pool-tasks 96 --expert-cache auto --vram-reserve-mib 1400`
(UD tiers: 896–1024), `--kv int8 --kv-resident 32768` (b5/b9-lane: k8v4/direct as stated), MTP + cyrillic
draft vocab, prompt-cache, conversation pool 24576 MiB ×2, `STRATA_KV_PREFETCH=1`; the full effective flag list
is in each folder's `config.json` and in the companion report. Our long-prompt instrument (53,326-token
corpus) and depth/battery tools are described in the report; they are nonstandard by design.

## Not measured / limits

Sustained thermal load, concurrency stage, >16K standard prompt points (32K/128K/256K), vision e2e, 786K/393K
UD tiers (bracketed by 524K and 1M), repeats beyond the protocol's n=3, headless runs (desktop was live),
DFlash2 on stock IQ3_S (docs keep it MTP for ISTA packs). The unlock is non-stock hardware: expect higher
prefill than a retail 4080 (SMs + ~20 % clocks) and near-equal decode (memory-bound). The b8 depth-probe note
and one 2-token-summary anomaly are stated as-is in the companion report (§D).

## Files

- `docs/bench-results/2026-11-20-standard-unlocked-rtx4080-{iq3s-b1, iq3s-b2-fused, iq3s-b4-alesha-cvec,
  sc117-b5, coder-b6, iq3s-b7-acvec, udq4-b8-262144, udq4-b9-262144-mtp}/` — results.json + README +
  sanitized config + engine log slice + outputs.jsonl (draft protocol ID labelled, per the submit rules).
- `docs/bench-results/2026-11-20-unlocked-rtx4080-night-report.md` — the human-readable companion: hardware
  evidence, ladder tables (nonstandard), control-vector batteries, pre-registered predictions vs measured Δ,
  and the full "not measured" list.

No site

Links install, modelos, releases.