Pull requests / #168

#168 hip: support Q2_K embeddings and type-aware PLE table rows

closed · @samuelishida · 0 comentarios · En GitHub

AMD / HIPNVIDIA / CUDAModels & quants

Descripción

Extends the HIP/CUDA quant paths to cover more public checkpoints:

- Bartowski's IQ2_XS uses Q2_K for `token_embd.weight`; add a GPU dequantizer and embedding gather, exposed through `iq_dequant_supported()` so it is accepted for embeddings but not silently treated as an MMVQ type.
- The AD IQ4_XS-M64 checkpoint stores PLE table rows as Q5_1; make row decode type-aware for Q4_0 / IQ4_NL / Q5_1 with matching row-byte counts in `PleReader` and its row cache.
- Add `hip_q2k_embed_parity` to exercise both synthetic and optional real IQ2_XS rows.

En el sitio

Enlaces a install, modelos, releases.