Issues / #1995
#1995 Request: Add Q4_1 PLE Dequantization Support
open · @dororo1989 · 1 commentaires · Sur GitHub
Setup & installAMD / HIPNVIDIA / CUDAModels & quantsWindows
Description
I am running Strata on Windows with an AMD Radeon RX 9070 (16 GB VRAM), 64 GB of system RAM, and the HIP backend. This setup makes it possible to run certain 4-bit quantized MoE models locally, even with relatively limited GPU memory. While testing the [[Qwen3.8 Flash-Next AD IQ4_XS M64 model](https://huggingface.co/AtomicChat/Qwen3.8-Flash-Next-GGUF)](https://huggingface.co/AtomicChat/Qwen3.8-Flash-Next-GGUF), I discovered that its `per_layer_token_embd.weight` tensor uses Q4_1 quantization and requires Q4_1 PLE dequantization support in Strata. With AI-assisted development, I implemented the missing Q4_1 PLE dequantization path and tested it against the actual GGUF data. The validation passed with bit-identical results. Would you consider integrating Q4_1 PLE dequantization support into Strata upstream? Full official M64 support would be welcome, but it is not a requirement for this request. Adding Q4_1 PLE support alone would already make the engine more compatible with models that use this format and would save users from maintaining custom patches across releases. I can provide the patch and validation details if helpful. Thank you for your work on Strata!
Sur le site
Liens install, modèles, releases.