贡献 / #1800
#1800 Support external PLE replacement for a missing split shard
open · @constantindjonkam · 0 评论 · 去 GitHub 看
说明
## What changed - Add opt-in `--skip-ple-shard N` support for split GGUF models when an external `strata-ple` file is supplied with `--ple-gguf`. - Keep default split validation strict: unrelated missing shards and inconsistent tensor counts still fail. - Validate the external PLE format and model geometry; when the skipped file is present, validate its header as the model's single PLE tensor. - Thread the omission through native embedding/head loading and `tools/iq_pack.py`. - Document the Swift Q4_K_L packing invocation. Example: ```bash python tools/iq_pack.py \ --gguf Swift-...-00001-of-00004.gguf \ --out packs/swift-q4kl \ --ple-gguf ple_fp8.gguf \ --skip-ple-shard 2 \ --compat-bf16 ``` The packer does not write `experts.bin` unless explicitly requested. The replacement interface is described generically as an external PLE artifact; the current example and validator use the existing FP8/E4M3 PLE representation and still validate its format and model geometry. ## Validation - `gguf_split_test`: passed, including accepted PLE-only omission and refused unrelated omission. - `python tools/test_iq_pack.py`: 25 passed. - CUDA Release build with MMQ/Q6 support: `strata` and `gguf_split_test` built successfully. - Packed the real Swift Q4_K_L model without `--experts-bin`. - Loaded and generated one token with the external FP8 PLE while shard 00002 was absent from the runtime directory (RTX 3090, GPU 0 only); output was produced successfully.
本站相关内容
相关页面的快捷入口。