#2750 kq_cpu_bench: quantize i-quant blobs with ggml_quantize_chunk (it crashed on them); KQ_PHASE=gu|down

本文

## kq_cpu_bench: i-quant blobs via ggml_quantize_chunk (it crashed on them); KQ_PHASE=gu|down

`kq_cpu_bench` builds its random experts with the CPU type traits' `from_float`, which is null for the i-quants. With
`--pairs iq2_s/iq4_nl` (or any IQ gate/up type) it segfaulted on the first pair, through a null function pointer, before
printing a number. `gdb`: `#0 0x0000000000000000 in ?? ()`, `#1 main ()`.

- `make_blob` now uses `ggml_quantize_chunk`, which quantizes every type. These i-quants need no importance matrix.
- `KQ_PHASE=gu` or `down` times only that half of each expert, and prints its GB/s over that half's bytes.

Measured on a Ryzen 7 3800X (Zen 2, AVX2), 1 thread, 512 MB of blobs, 1 token:

| Format | Gate/up only | Down only |
| --- | ---: | ---: |
| IQ2_S | 4.3 GB/s | - |
| IQ3_XXS | 3.9 GB/s | - |
| IQ3_S | 4.0 GB/s | - |
| IQ4_NL | - | 9.9 GB/s |

The bench's other pairs and its default output are unchanged.

🤖 Generated with [Claude Code](https://claude.com/claude-code)