Pull requests / #2031

#2031 Add Q4_1 PLE dequantization support

open · @dororo1989 · 0 Kommentare · Auf GitHub

AMD / HIPNVIDIA / CUDAModels & quantsWindows

Beschreibung

## Summary
- Add Q4_1 block dequantization for PLE, using 20-byte blocks and 100-byte rows.
- Register Q4_1 in the table-driven PLE format registry.
- Add row geometry and format registration checks.
- Compare Strata's Q4_1 dequantization against GGML's `dequantize_row_q4_1`.

## Validation
- Windows HIP build tested on AMD Radeon RX 9070 (gfx1201).
- `ple_reader_test.exe --selftest` passed, including 2,048 blocks and 2,048 rows compared against GGML with maximum absolute error of `0.000e+00`.
- Loaded and ran the Qwen3.8 Flash-Next AD IQ4_XS model with the custom HIP build.
- No GPU-side PLE reader changes were made. CUDA and SYCL were not tested separately.

I used AI assistance during development, so there may be things I have overlooked. If anything needs improvement or further testing, please let me know. I would be happy to help make any necessary changes.

Thank you again for your time and for maintaining this project!

Mehr auf der Site

Links zu Install, Modellen, Releases.