问题 / #2726

#2726 [Feature]: Offline conversion of FP4-Local CSF checkpoints

open · @hagope · 0 评论 · 去 GitHub 看

Setup & installNVIDIA / CUDA

说明

## What do you want to do?

Convert the text weights of [`nvidia/Qwen3.8-Flash-Next-FP4-Local`](https://huggingface.co/nvidia/Qwen3.8-Flash-Next-FP4-Local) for Strata's existing NVFP4 expert path.

This checkpoint stores routed-expert block scales as CSF fixed/exception streams, dense weights as mixed MXFP8/NVFP4, and the PLE table as NVFP4. The existing `nvfp4_convert.py` path does not decode these streams and formats. The shard index also places files under `tensors/`.

The proposed change is an explicit offline converter: losslessly expand/repack routed-expert scales, decode dense weights before the existing BF16/F16/Q8_0 policy, and stream the PLE table to the existing BF16 reader. The installer, runtime, default converter and engine backends stay unchanged. Vision and MTP conversion are outside this change.

A bounded experiment has already checked nine real expert projections against an uncompressed reference and the pinned converter, and three converted expert blobs passed Strata's CUDA expert parity test on an RTX 5090. Full-model serving, answer quality and throughput remain untested. A draft PR with CPU tests and the bounded real-tensor conversion check will follow.

Related format work: [vLLM #60646](https://github.com/vllm-project/vllm/pull/60646). [Strata #1859](https://github.com/Niko1221/Strata/pull/1859) is a separate native safetensors source; this proposal uses offline conversion instead of adding a native weight backend.

本站相关内容

相关页面的快捷入口。