Pull requests / #2233

#2233 docs(bench): RTX 3060 12GB / 32GB RAM Coder IQ1_M with Budget12 64K profile

open · @wookie4lif3 · 0 comments · View on GitHub

BenchmarksSetup & installNVIDIA / CUDAModels & quantsDocumentationWindows

Description

# docs(bench): RTX 3060 12GB / 32GB RAM Coder IQ1_M with Budget12 64K profile

## Summary

Adds a community results-only report from one Windows 11 / RTX 3060 12GB / i5-11400F / 31.8 GiB RAM machine running **unmodified upstream Strata v0.1.43** (`7695859aa13afb248bdb4d36f9b2e95707db079d`) with Qwen3.8-Flash-Next **Coder IQ1_M**.

The contribution has three **separately labeled** evidence groups: the original upstream 32K speed suite, non-stock near-32K extensions, and a distinct 65,536-token Budget12 configuration and one-shot reasoning investigation. It does **not** change any Strata source or model weights.

## Stock and custom 32K measurements

- Stock `tools/strata_bench.py` (3 repeats): **33/33** completed requests; short-chat decode median **26.0 tok/s** (9 requests); fresh 4K prefill **717.0 tok/s**; fresh 16K prefill **904.5 tok/s**. Original `results.json` included.
- Custom near-32K extension: **31,294** actual fresh input tokens, **908.4 tok/s** median prefill and **34.56 s** median first-token latency (n=3). This is **not** an exact 32,768-token input.
- Built-in near-30.5K needle check: **3/3 found**, one attempt each at 10%, 50%, and 90% positions. A predictable 256-token sequence achieved **57.2 tok/s** median decode (n=3); **not** a general chat-speed claim.

## Separate experimental Budget12 64K profile

- **Actual, SHA-256-verified local JSON exported as a sanitized, non-runnable path-placeholder config**, with the original source and sanitized-output hashes recorded in `budget12-export-receipt.json`. New files `budget12-sanitized-config.json` and `budget12-REPRODUCE.md` document the exact observed flags and how to adapt paths on a different installation.
- Config: `--max-context 65536`, `--kv int8` (non-streaming), `--expert-cache auto`, `--prefill auto`, `--spec 4`, `--spec-min-p 0.5`, and `--resident-budget-gib 12`. The verified original configuration was **not modified**, and the upstream engine was **not forked**. Budget12 run reported **2,152 expert cache slots**.
- Full-resident 64A finished with ~451 MiB free host RAM; 64B stopped at the safety preflight. Budget12 enabled completed ~54K-input runs with much more sampled host RAM available, with different throughput and task results.
- Exact-answer diagnostics (single-shot cases): 64F **10/10** manual repository evaluation, 64G **12/13**, 64H **accuracy not evaluable** (no final answer; `length`), 64I **12/18**, 64J **24/24** (including six canonical/replay record-provenance fields). The per-request `reasoning_budget_tokens: 6144` and `max_tokens: 8192` produced valid completions for 64I/J.

## Evidence and limitations

- Stock `results.json`, custom `extended-results.json`, needle results, the exact custom benchmark script and separate 64K summary CSV are included. The synthetic 64K grades are self-contained research tests summarized in the report; private case packets and full reasoning transcripts are **not** included.
- All tests are from **one host**. 64I and 64J differ in randomized input and prompting, so their score difference **does not demonstrate a controlled causal improvement** or general 100% reliability. The 64J prefill rate slowed to **349.2 tok/s** from 64I's **833.5 tok/s**, with the cause unknown.
- The sanitized JSON intentionally contains invalid local-path placeholders. It is a verified configuration **template for manual adaptation, not a drop-in one-click installation**. RAM/VRAM safety thresholds were enforced by an external benchmark runner, not Strata itself. No hardware-independent or production-grade correctness claim is made.

Please review as a **documentation-and-community-benchmark contribution** following `docs/COMMUNITY_BENCHMARKS.md`. No runtime changes, binaries, copyrighted model weights, private machine paths or raw user test transcripts are proposed.

Related on strata.com

Editorial links to help you install, pick models, or read release notes — not part of the upstream thread.