#2727 tools: add opt-in FP4-Local CSF checkpoint conversion

本文

The FP4-Local checkpoint stores expert scales as CSF streams and uses mixed MXFP8/NVFP4 weights, which the existing offline converter cannot read. This adds a separate explicit converter that writes a main text GGUF and a streamed BF16 PLE GGUF for Strata’s existing readers.

Refs #2726. The full pinned checkpoint has now been converted and served through Strata 0.1.44.1 on an RTX 5090 32 GB / Core Ultra 9 285K / 64 GiB RAM / Ubuntu host.

- Routed-expert codes and decoded scale bytes are retained; the pinned converter performs the NVFP4 repack.
- Dense MXFP8/non-expert NVFP4 weights are decoded before the existing BF16/F16/Q8_0 policy, adding rounding.
- The NVFP4 PLE table is streamed to BF16 in bounded row chunks (102,400,491,520 payload bytes for FP4-Local).
- Metadata-only `--dry-run` reports the conversion plan. Outputs/manifests are new files; source checkpoint files remain read-only.
- Experts are grouped by layer before repacking so the real checkpoint’s interleaved shards do not accumulate the entire arena in RAM. Explicit source config values are preserved across AutoConfig normalization, including `full_attention` names needed by the pinned metadata writer. Both full-run failures have regression tests with failing negative controls against the previous implementation.
- Vision and MTP are excluded. The upstream installer, runtime/backend and ordinary `nvfp4_convert.py` remain unchanged. Compatibility adapters are scoped to the new command and restored on exit/failure.

Validation:

- CPU suites: 5 codec tests + 14 converter tests passed on the target host; Ruff and `git diff --check` passed.
- The CLI also converted a bounded real-tensor fixture end to end, retaining nine independently computed expert projection hashes and writing a readable BF16 PLE file.
- Full output: 1,511 tensors / 73,939,912,352 bytes in the main GGUF; 144 NVFP4 expert tensors with 512 experts each. Nine projection hashes/global scales matched; 18 BF16 PLE rows across first/middle/final shards matched source reconstruction. Expert pack: 48 layers / 63.28 GiB.
- The initial full export’s fixed-size compression-ratio metadata was repaired without changing tensor payloads. All 38 architecture fields then matched a fresh metadata export from the corrected converter, and tensor checks were repeated.
- Live API checks passed text, structured tool calling and tool-result continuation, separated reasoning, streaming and retrieval from an 18,040-token prompt. A client gateway round trip passed. The corrected service remained stable with zero restarts during these checks.
- Smoke-check rates: 3,224 prompt tokens/s on the retrieval prompt, 49 tokens/s for a 77-token reasoning reply, 54 tokens/s for a 109-token streamed paragraph. Configuration: 128K maximum context, INT8 KV with 32K resident cells, 38 GiB expert RAM budget, no MTP or vision sidecar. These short checks are not a quality/throughput comparison; the full 128K limit was not exercised.
- Earlier three-expert CUDA parity passed the existing test (1.096%, 1.156%, 1.159% relative output error; threshold 3%). No GPU/kernel changes.

The docs record the pinned checkpoint/reference revisions, precision changes, memory needs, conversion caveats and measurements. Full answer-quality comparison, full-context retrieval, vision and MTP remain untested.

Format reference: [vLLM #60646](https://github.com/vllm-project/vllm/pull/60646). AI-assisted implementation and testing.