Pull requests / #168

#168 hip: support Q2_K embeddings and type-aware PLE table rows

closed · @samuelishida · 0 评论 · 在 GitHub 查看

AMD / HIPNVIDIA / CUDAModels & quants

描述

Extends the HIP/CUDA quant paths to cover more public checkpoints:

- Bartowski's IQ2_XS uses Q2_K for `token_embd.weight`; add a GPU dequantizer and embedding gather, exposed through `iq_dequant_supported()` so it is accepted for embeddings but not silently treated as an MMVQ type.
- The AD IQ4_XS-M64 checkpoint stores PLE table rows as Q5_1; make row decode type-aware for Q4_0 / IQ4_NL / Q5_1 with matching row-byte counts in `PleReader` and its row cache.
- Add `hip_q2k_embed_parity` to exercise both synthetic and optional real IQ2_XS rows.

站内延伸阅读

链到安装、模型与版本说明,便于 SEO/GEO,非官方 issue 正文。