#2333 Community benchmark: RTX 3060 12GB - IQ3_XXS vs IQ3_S on engine 0.1.44

本文

# Community benchmark on NVIDIA GeForce RTX 3060 — IQ3_XXS vs IQ3_S on engine 0.1.44

Measured on 2026-10-14 with `tools/strata_bench.py` (suite 1, 3 runs per test, medians with ranges).
Speeds are the engine's own timings; first-token times are measured by the client over HTTP.
Speed only: no correctness check was run.

Two quantization sizes of the same Qwen3.8-Flash-Next GSQ-RCO pack, measured back to back on the same
machine and the same server settings. Engine 0.1.44's release notes state that answers from the GSQ-RCO
packs are byte-identical to 0.1.43.1 in the project's identity gates, so the two runs below are directly
comparable to each other.

## Hardware and software

- GPU and VRAM: NVIDIA GeForce RTX 3060, 12 GiB VRAM, power limit 170.00 W (stock, unmodified)
- CPU: Intel Xeon E-2186G, 6 cores / 12 threads @ 3.79 GHz max; installed RAM: 63.8 GiB
- Storage: 1x Samsung MZVLB512HAJQ-000L7, 477 GB NVMe SSD (the only disk; model and arena both on it)
- OS: Windows 11 (10.0.26300); GPU driver: 617.42
- Strata: project 0.1.44, commit ed4773eb; engine 0.1.44 (ready-made Windows x64 build, sm_86)
- Background workloads during the runs: the Strata server itself (~42 GB resident), a browser
  (~0.9 GB) and a monitoring agent (~0.8 GB); ~10.7 GiB of RAM left free with IQ3_XXS loaded.
  No other workload was started for the measurement.

## Model and configuration (identical for both sizes)

- Model: Qwen3.8-Flash-Next (GSQ-RCO), quantization IQ3_XXS or IQ3_S
- Context: 262144 tokens; `--kv int8 --kv-resident 32768 --spec 4 --spec-min-p 0.5`
- MTP draft layer enabled; `--expert-cache auto --prefill auto`
- **Non-default:** `--vram-reserve-mib 100` instead of the default 700. It leaves 600 MiB more
  VRAM for the expert cache, about 12% larger (~38 more expert slots). The other engine
  arguments in the line above are setup's own defaults. Both sizes were measured with the same
  value, so the comparison between them is unaffected; a run at the default 700 would decode a
  little slower.
- parallel slots: 1; thinking off for the suite; temperature 0

## Method

Run by `python tools/strata_bench.py` against the HTTP API of a Strata server (already running); model
loading is not timed. One warm-up chat and a 1K prompt are discarded first. Every prompt starts with a
unique tag, so the conversation cache cannot help the fresh-prompt tests.

- short chat: the three questions of `tools/ab_engine.py`, up to 100 new tokens, 3 runs
- 4K / 16K prompts: generated word lists sized by a calibration request, 24 new tokens, 3 fresh prompts each
- agent session: system prompt and four tools, then 5 turns that each add about 1500 words of a project
  file and continue from the model's own earlier answers (the way agent clients resend a conversation),
  64 new tokens per turn, 3 sessions

## Results

Medians, ranges over the runs. "Delta" is IQ3_XXS relative to IQ3_S.

| Configuration | IQ3_S | IQ3_XXS | Delta |
| --- | ---: | ---: | ---: |
| short chat, decode tok/s | 29.2 (25.8-38.0) | **34.7** (30.3-38.5) | **+18.8%** |
| short chat, first token (s) | 1.18 (1.03-1.27) | **1.04** (0.93-1.14) | -11.7% |
| 4K prompt, prefill tok/s | 806.0 (795.6-806.2) | **883.0** (878.3-886.9) | **+9.6%** |
| 4K prompt, first token (s) | 5.06 (5.03-5.12) | **4.60** (4.57-4.64) | -9.0% |
| 16K prompt, prefill tok/s | 958.7 (957.3-959.0) | **1049.5** (1048.1-1051.2) | **+9.5%** |
| 16K prompt, first token (s) | 16.84 (16.81-16.87) | **15.38** (15.34-15.38) | -8.7% |
| agent turns, decode tok/s | 26.9 (17.7-39.7) | **37.1** (24.5-42.3) | **+37.9%** |
| agent turns, first token (s) | 5.06 (4.77-5.25) | **4.38** (4.06-4.56) | -13.5% |
| agent conversation cache hit | 68.73% (68.44-68.76) | 68.62% (68.56-68.72) | -0.2% (level) |
| total suite wall time (s) | 227.9 | **199.2** | -12.6% |

### Agent session detail (run 1, IQ3_XXS)

| Turn | Prompt tokens | Reused | Read | Prompt tok/s | First token (s) | Decode tok/s |
| ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| 0 | 609 | 0 | 609 | 243.9 | 2.535 | 20.1 |
| 1 | 3741 | 621 | 3120 | 710.3 | 4.438 | 34.5 |
| 2 | 5274 | 3804 | 1470 | 366.0 | 4.057 | 35.2 |
| 3 | 8718 | 5338 | 3380 | 753.2 | 4.555 | 38.8 |
| 4 | 11625 | 8763 | 2862 | 663.2 | 4.376 | 40.3 |
| 5 | 14541 | 11642 | 2899 | 668.5 | 4.4 | 25.1 |

The IQ3_S turn-by-turn numbers are the same shape (609 / 3120 / ~1500 / ~3400 / 2862 / 2899 tokens
read per turn). The suite's "earlier-turn tokens read again per session" metric reports medians of 3
for IQ3_S and 74 for IQ3_XXS, but the raw per-turn reads differ by only ~70 tokens per session in
total, so we treat that metric as level between the two, not as an IQ3_S advantage.

## Engine state reported by the server at the time of the run

| | IQ3_S | IQ3_XXS |
| --- | ---: | ---: |
| expert arena | 47,962 MiB | 40,925 MiB |
| expert slots | 2,477 | 3,142 |
| expert cache (VRAM) | 4,869 MiB | 5,251 MiB |
| free VRAM after load | 0 MiB | 0 MiB |
| Strata process resident | 48.3 GiB | 42.5 GiB |
| system RAM free | 4.7 GiB | 10.7 GiB |

Both sizes exhaust the 12 GiB card. IQ3_S's larger arena is mostly RAM-resident, and with less room
left for the VRAM expert cache it loses the decode lead it would otherwise have from its higher
precision.

## Correctness and limitations

- Speed suite only; no correctness or quality check was run. Which size "answers better" is not
  measured here — only how fast each one answers.
- The conversation uses the model's own replies, so token counts differ a little between runs.
- No concurrency test (`--clients`), no 32K prompt test (`--long`) in this round.
- Both runs share the same expert profile file, which was learned under IQ3_XXS usage; IQ3_S was not
  re-calibrated before its run.
- Background load and power limits are as stated above; the GPU was at its stock 170 W limit.

## Conclusion

On a 12 GiB RTX 3060 with 64 GiB of RAM, IQ3_XXS is faster than IQ3_S on every test in the suite,
by 9-19% on prompts and 19-38% on decode, and it leaves twice the free RAM. IQ3_S's arena does not
fit the card either way, so the extra precision does not pay for itself here. Engine 0.1.44's setup
menu now recommends IQ3_S as the 64 GB default, which this measurement does not support on 12 GiB
of VRAM.

Where IQ3_S starts to win is not measured here: 12 GiB is the only VRAM this machine has. We would
expect it from about 16 GiB up, where the larger expert cache keeps more of the experts in VRAM, but
that is an expectation from the cache-size arithmetic, not a result. The measured conclusion holds
for 12 GiB; cards above that need their own run.

---

Per-request data: `results.json` in each run's folder. Home folders, user and host names and IP
addresses are already replaced.