Pull requests / #3597

#3597 [Benchmark] RTX 3070 Ti 8 GB + Ryzen 5 7600X — Qwen3.6-35B-A3B abliterated Q5_K_S (0.1.44.1)

open · @jkir4n · 0 comments · View on GitHub

BenchmarksSetup & installAMD / HIPNVIDIA / CUDAModels & quantsDocumentationWindows

Description

Supersedes #2581 (closed in the 0.2.0-beta tracker reset; branch rebased onto current `main`).

**Area:** [Benchmark]

## Problem and result

There was no community datapoint for Qwen3.6-35B-A3B on an 8 GiB consumer card. This adds one: a Strata 0.1.44.1 run on an RTX 3070 Ti + Ryzen 5 7600X with a community abliterated Q5_K_S build substituted into the install slot. Before: no result in `bench/results/` for this hardware class. After: a result directory in the same shape as the existing community entries.

## Related discussion

none

## Approach

Two files, both new, no code touched:

- `bench/results/2026-10-15-community-rtx-3070-ti-qwen3-6-35b-a3b-ud-q4-k-xl/README.md` — hardware and software, model and configuration, method, results, correctness notes, limits.
- `…/results.json` — per-request data from the suite.

Headline numbers (medians, 3 runs): short chat decode 50.6 tok/s (46.6–60.7); 4K fresh prompt 1,189 tok/s prompt / 3.40 s TTFT; 16K fresh prompt 1,100 tok/s prompt / 14.62 s TTFT; 32K-context agent session decode 35.5 tok/s (22.4–47.3) with TTFT 8.28 s (3.31–14.05). VRAM steady at 7,666 of 8,192 MiB; all routed experts run from pageable host RAM at this quant.

## Validation

**Revision:** rebased onto `main` at `011c9e60` (the 0.2.x line). This PR is documentation-only — no code, no runtime path — so the checks that apply are the added JSON parsing and the file shape.

**Passed:** `python -m json.tool results.json` on the added file (valid). The benchmark itself was run with `python tools/strata_bench.py` against a live HTTP server (model load not timed, one warm-up plus a 1K prompt discarded, unique prompt tags so the conversation cache cannot help). Results parsed from the engine's own timings; per-request data in `results.json`. A separate 5/5 correctness battery (arithmetic, the five-widgets puzzle, executed fizzbuzz, capital of Australia, exact bullet format) is recorded in the README, not part of the suite.

**Failed:** none observed.

**Not run:** no rebuild of the engine (official Windows CUDA 13 release binary used as-is); no CUDA/HIP/SYCL/Metal variants; single machine, single session; no correctness suite integrated into the benchmark.

## Limits

Numbers are single-session medians on a box at its RAM edge (free RAM ranged 0.2–3.8 GiB during the runs; chat decode dips to ~14 tok/s when Windows trims model pages), so expect variance with different background load. The model is a community abliterated fine-tune (Q5_K_S, MTP block preserved) substituted into the UD-Q4_K_XL slot — the server label comes from that slot. A second suite run and a 4-client concurrency run on the same day are documented in the README as additional measurements, not part of the primary table.

Related on this site

Quick links to related pages.