Pull requests / #2233
#2233 docs(bench): RTX 3060 12GB / 32GB RAM Coder IQ1_M with Budget12 64K profile
open · @wookie4lif3 · 0 commentaires · Sur GitHub
BenchmarksSetup & installNVIDIA / CUDAModels & quantsDocumentationWindows
Description
# docs(bench): RTX 3060 12GB / 32GB RAM Coder IQ1_M with Budget12 64K profile ## Summary Adds a community results-only report from one Windows 11 / RTX 3060 12GB / i5-11400F / 31.8 GiB RAM machine running **unmodified upstream Strata v0.1.43** (`7695859aa13afb248bdb4d36f9b2e95707db079d`) with Qwen3.8-Flash-Next **Coder IQ1_M**. The contribution has three **separately labeled** evidence groups: the original upstream 32K speed suite, non-stock near-32K extensions, and a distinct 65,536-token Budget12 configuration and one-shot reasoning investigation. It does **not** change any Strata source or model weights. ## Stock and custom 32K measurements - Stock `tools/strata_bench.py` (3 repeats): **33/33** completed requests; short-chat decode median **26.0 tok/s** (9 requests); fresh 4K prefill **717.0 tok/s**; fresh 16K prefill **904.5 tok/s**. Original `results.json` included. - Custom near-32K extension: **31,294** actual fresh input tokens, **908.4 tok/s** median prefill and **34.56 s** median first-token latency (n=3). This is **not** an exact 32,768-token input. - Built-in near-30.5K needle check: **3/3 found**, one attempt each at 10%, 50%, and 90% positions. A predictable 256-token sequence achieved **57.2 tok/s** median decode (n=3); **not** a general chat-speed claim. ## Separate experimental Budget12 64K profile - **Actual, SHA-256-verified local JSON exported as a sanitized, non-runnable path-placeholder config**, with the original source and sanitized-output hashes recorded in `budget12-export-receipt.json`. New files `budget12-sanitized-config.json` and `budget12-REPRODUCE.md` document the exact observed flags and how to adapt paths on a different installation. - Config: `--max-context 65536`, `--kv int8` (non-streaming), `--expert-cache auto`, `--prefill auto`, `--spec 4`, `--spec-min-p 0.5`, and `--resident-budget-gib 12`. The verified original configuration was **not modified**, and the upstream engine was **not forked**. Budget12 run reported **2,152 expert cache slots**. - Full-resident 64A finished with ~451 MiB free host RAM; 64B stopped at the safety preflight. Budget12 enabled completed ~54K-input runs with much more sampled host RAM available, with different throughput and task results. - Exact-answer diagnostics (single-shot cases): 64F **10/10** manual repository evaluation, 64G **12/13**, 64H **accuracy not evaluable** (no final answer; `length`), 64I **12/18**, 64J **24/24** (including six canonical/replay record-provenance fields). The per-request `reasoning_budget_tokens: 6144` and `max_tokens: 8192` produced valid completions for 64I/J. ## Evidence and limitations - Stock `results.json`, custom `extended-results.json`, needle results, the exact custom benchmark script and separate 64K summary CSV are included. The synthetic 64K grades are self-contained research tests summarized in the report; private case packets and full reasoning transcripts are **not** included. - All tests are from **one host**. 64I and 64J differ in randomized input and prompting, so their score difference **does not demonstrate a controlled causal improvement** or general 100% reliability. The 64J prefill rate slowed to **349.2 tok/s** from 64I's **833.5 tok/s**, with the cause unknown. - The sanitized JSON intentionally contains invalid local-path placeholders. It is a verified configuration **template for manual adaptation, not a drop-in one-click installation**. RAM/VRAM safety thresholds were enforced by an external benchmark runner, not Strata itself. No hardware-independent or production-grade correctness claim is made. Please review as a **documentation-and-community-benchmark contribution** following `docs/COMMUNITY_BENCHMARKS.md`. No runtime changes, binaries, copyrighted model weights, private machine paths or raw user test transcripts are proposed.
Sur le site
Liens install, modèles, releases.