Issues / #2744

#2744 [FEATURE] QAD Model? - Nvidia's Quantization-Aware Distillation

open · @SabinStargem · 0 comments · View on GitHub

Models & quants

Description

I don't really know the details, but apparently this allows for an improvement over 4-bit QAT.   Two links below, one to the source of the model, and another to the GGUF.   Weighs in at 106gb, probably requires Blackwell hardware.

[nvidia/Qwen3.8-Flash-Next-FP4-Local](https://huggingface.co/nvidia/Qwen3.8-Flash-Next-FP4-Local)

[fairfieldt/Qwen3.8-Flash-Next-NVFP4-QAD-GGUF](https://huggingface.co/fairfieldt/Qwen3.8-Flash-Next-NVFP4-QAD-GGUF)

Related on this site

Quick links to related pages.