#1850 Community report: 9-file quant battery on RTX 5070 Ti 16 GB (ISTA vs Swift vs Gyro-S, engine 0.1.40.3)

Description

### What this is

A full community benchmark report requested via Xavier: **9 files, 3 labs** — ISTA-DASLab GSQ-RCO (IQ3_XXS / Q2_0 / IQ2_XS / IQ3_S), UkisAI Swift-1.5 (the same four), and AgentionAI Gyro-S TQ1_0 — all on **one rig: RTX 5070 Ti 16 GB, 64 GB RAM, Windows + WSL**, with the engine pinned to **0.1.40.3 (digest `34CDE150B21148E6`)** for every Strata run (Gyro-S on its own `agentionai/Strata` rc1 engine).

Battery: HumanEval+ 164 · tool-eval-bench 92 agentic scenarios **× 5 runs per arm** · five-bugs debugging (25 sessions) · GSM8K / MMLU / IFEval · needle @260K · llama-benchy at 0/32/128K context. **18 arms (file × effort), ~1,900 measured runs.**

The full visual report is [`strata-community.pdf`](https://github.com/Zauberio/Strata/raw/study/quant-battery-rtx5070ti/bench/results/2026-10-10-community-quant-study-rtx5070ti/strata-community.pdf) (direct download) (14 pp); the GitHub-rendered version is the folder's [`README.md`](https://github.com/Zauberio/Strata/blob/study/quant-battery-rtx5070ti/bench/results/2026-10-10-community-quant-study-rtx5070ti/README.md). Every number in both is machine-extracted from [`data/ALL-METRICS.json`](https://github.com/Zauberio/Strata/blob/study/quant-battery-rtx5070ti/bench/results/2026-10-10-community-quant-study-rtx5070ti/data/ALL-METRICS.json) — nothing is hand-typed, and `scripts/` rebuilds every figure from the raw JSONs.

### Headlines (all measured; details in the README)

- **Coding ceiling is shared.** 95.7–96.3 HE+ is reachable **six ways** — the fight is cost, not ceiling. Swift IQ3_S low reaches it in **15m**; Qwen IQ3_S xhigh takes 67m.
- **File size class beats quant name on speed.** 66–68 GB files decode at **129–137 tok/s**; the 76–84 GB files at **91–110** on 16 GB (files bigger than VRAM stream experts from RAM).
- **Effort is part of the model you pick.** Swift IQ3_S ranks #1 all-rounder at low effort, #10 at xhigh. On two arms, xhigh makes agents act bolder → safety-trap scenarios score **0 in all 5 runs**.
- **Reliability ≠ mean.** Pass^5 floors (full marks in all 5 runs) sit up to 18 points under the mean; best floor is Swift Q2_0 low (78.3).
- **Needle 100% @260K** for all eight Strata-served files — quant depth is irrelevant to retrieval. Gyro-S's serve caps at 131K (100% there).

### Engine findings (for Niko / AgentionAI)

- **C1 — rc1 fork has no tool support.** Served directly on WSL loopback (no proxy): plain OK · tools-only OK · tools+choice OK · **forced `tool_choice` → `finish=stop`, `tool_calls=NONE`**. The serve accepts `tools` params and silently drops them. This blocks Gyro-S from agentic eval; quality is fine (96.3 HE+, 92 GSM8K, needle 100% at its cap) — the gap is plumbing, not the quant. Full A/B/C/D repro in the README.
- **C2 — Windows→WSL LAN forwarding resets long/streaming POSTs** (root cause of a "crash" we first misdiagnosed). Loopback + SSH reverse tunnel is clean; a 90-line compat proxy is the reference workaround.
- **C3 — `--vram-reserve-mib 1212`** (the engine's own computed number) kills the 0-MiB-VRAM crash class under WSL contention.
- **C5 — 0.1.40.3 held up:** 5 campaigns, zero infra failures after known fixes; auto-update rollback protected our pin through two failed v0.1.41 boots (FYI).
- **C6 — native_experts v4 mapping is deterministic:** Swift packs 96/96 tensors exact (max |err| 0); ISTA quants 1080/1080 direct.

### What we did NOT test

Vision quality · multi-GPU · 24/32 GB cards (**every number here is 16 GB**) · KLD vs vendor cards · long multi-day agent sessions · seeds other than 42 · Coder / IQ1_M files. VRAM-scaling statements in the PDF are labeled expectation, not data.

### We publish our fixes

A claim-by-claim audit the day before posting corrected **15 claims** — e.g. safety chips said "capped ★★" (the true cap is ★★★, and only 8/16 arms are capped, not all); a hero stat paired two walls that were never equal-quality; "all nine pass needle at 260K" when Gyro-S's serve caps at 131K. The corrections log is in the PDF and README. If you find a 16th, open an issue on this PR — we'd rather be corrected than polished.

### Reproduce it

`scripts/extract_all.py → data/ALL-METRICS.json → make_charts.py (F1–F15) → compile_pr_readme.py`. The two figure/README scripts run from the package alone (`STUDY_BASE` defaults to the package root); `extract_all.py` runs on the study rig. Engine pinned `34CDE150B21148E6` · Strata `0.1.40.3` · tool-eval-bench `2.7.1.dev16+gf7c34a130`.