Pull requests / #2505
#2505 setup: a shard with no tensors is not "short" (UD-Q5_K_XL 00001-of-00006 stopped setup on a correct download)
open · @joesinvestments · 0 commentaires · Sur GitHub
Setup & installAMD / HIPNVIDIA / CUDAModels & quants
Description
## Title Issue: none open; found while setting up UD-Q5_K_XL on a Mac (`./setup.sh --setup --family unsloth --model UD-Q5_K_XL`). ## Summary All six UD-Q5_K_XL shards download, then setup stops: `Qwen3.8-Flash-Next-UD-Q5_K_XL-00001-of-00006.gguf is short: 10,946,618 of 10,946,624 bytes (6 missing)`. The file is right: the Hub lists it at 10,946,618 bytes and its SHA-256 equals the pin (`610922cd...`). That shard holds only metadata, and `check_shards()` takes `g.data_start` (the header rounded up to the alignment) as its least length, which a tensor-less file never reaches. A shard with no tensors has no tensor data to be short of. ## What changed - `setup.py`, `check_shards()`: `if have < need:` becomes `if have < need and g.tensors:`. - `tools/test_shards.py`: `test_tensorless_shard_needs_no_alignment_padding` (a 24-byte GGUF with no tensors passes). - Left alone: shards with tensors are checked as before (the short-shard and #657 tests are unchanged and pass); the pinned SHA-256 checks still cover the tensor-less shard. ## How it was checked - `python -m unittest tools.test_shards`: the new test fails on `main` 296b59d2 with the "is short" message and passes with the change; 14 of 14 pass. - `tools/test_setup_*.py`, all 40, on `main` and on this branch: every test ends with the same result on both (run on macOS, where 15 of them fail on `main` too for PC-only reasons such as the ROCm wheels). - End to end on a Mac Studio M5 Max 64 GB (macOS 27.0.1): with this change setup finished for UD-Q5_K_XL, and the engine serves it (Metal, experts streamed from the SSD). - Not built: no engine code is touched.
Sur le site
Liens install, modèles, releases.