Pull requests / #64

#64 parity tool: a new-kernel mismatch must fail the run; dispatch comment fallback was stale

closed · merged 2026-09-28 · @pipeob0 · 0 comentarios · En GitHub

Models & quants

Descripción

The two follow-ups you listed in your #43 merge note. Both are yours to shape; nothing here changes a kernel.

**1. The comment above the gate/up dispatch described a fallback that no longer exists.** It said `STRATA_NO_IQ512 / STRATA_NO_IQ256 fall back to ggml-cpu's single-token vec_dot`. Since #43, `STRATA_NO_IQ512` drops an AVX-512 CPU to the AVX-2 kernel, `STRATA_NO_IQ256` drops the AVX-2 kernel, and ggml's `vec_dot` is reached only with **both** set (and on a CPU without AVX-512, `STRATA_NO_IQ512` changes nothing at all). Comment-only commit.

**2. The parity tool printed the new kernels' distance from ggml's `vec_dot` without affecting the exit code.** A kernel that silently diverged still reported `0 failures`, which is the one property of that tool people rely on. Both comparisons (the gate/up rows in `check`, and the IQ4_NL down) are now gated at 1e-5 and increment `failures`. Measured headroom: gate/up rows 3.1-3.4e-08, IQ4_NL down ~1e-7, so the gate sits ~100x above the observed spread.

Verified on this box (IQ2_XS, IQ2_XXS, IQ3_S, IQ4_NL layers):

- threshold 1e-5 → `native_expert_parity: 0 failures` (unchanged, as it should be);
- threshold temporarily dropped to 1e-9, i.e. below the observed spread, to prove the gate is live:

```
iq2_xs  avx2 gate rows MISMATCH (rel 3.12e-08 > 1e-9)
iq4_nl  down           MISMATCH (rel 9.91e-08 > 1e-9)
iq2_xxs avx2 gate rows MISMATCH (rel 3.35e-08 > 1e-9)
iq3_s   avx2 gate rows MISMATCH (rel 3.34e-08 > 1e-9)
```

Both sites fire, and the run exits non-zero. The threshold is back at 1e-5 in the commit.

Left alone deliberately: the `q2_0` down comparison prints ~2.1e-04, which is the pre-existing Q2_0 kernel against ggml and is expected to be that coarse — gating it at 1e-5 would be a false alarm, so it stays print-only.

En el sitio

Enlaces a install, modelos, releases.