Issues / #1995

#1995 Request: Add Q4_1 PLE Dequantization Support

open · @dororo1989 · 1 comments · View on GitHub

Setup & installAMD / HIPNVIDIA / CUDAModels & quantsWindows

Description

I am running Strata on Windows with an AMD Radeon RX 9070 (16 GB VRAM), 64 GB of system RAM, and the HIP backend. This setup makes it possible to run certain 4-bit quantized MoE models locally, even with relatively limited GPU memory.

While testing the [[Qwen3.8 Flash-Next AD IQ4_XS M64 model](https://huggingface.co/AtomicChat/Qwen3.8-Flash-Next-GGUF)](https://huggingface.co/AtomicChat/Qwen3.8-Flash-Next-GGUF), I discovered that its `per_layer_token_embd.weight` tensor uses Q4_1 quantization and requires Q4_1 PLE dequantization support in Strata.

With AI-assisted development, I implemented the missing Q4_1 PLE dequantization path and tested it against the actual GGUF data. The validation passed with bit-identical results.

Would you consider integrating Q4_1 PLE dequantization support into Strata upstream?

Full official M64 support would be welcome, but it is not a requirement for this request. Adding Q4_1 PLE support alone would already make the engine more compatible with models that use this format and would save users from maintaining custom patches across releases.

I can provide the patch and validation details if helpful.

Thank you for your work on Strata!

Related on strata.com

Editorial links to help you install, pick models, or read release notes — not part of the upstream thread.