Pull requests / #2505

#2505 setup: a shard with no tensors is not "short" (UD-Q5_K_XL 00001-of-00006 stopped setup on a correct download)

open · @joesinvestments · 0 comentarios · En GitHub

Setup & installAMD / HIPNVIDIA / CUDAModels & quants

Descripción

## Title
Issue: none open; found while setting up UD-Q5_K_XL on a Mac (`./setup.sh --setup --family unsloth --model UD-Q5_K_XL`).

## Summary
All six UD-Q5_K_XL shards download, then setup stops:
`Qwen3.8-Flash-Next-UD-Q5_K_XL-00001-of-00006.gguf is short: 10,946,618 of 10,946,624 bytes (6 missing)`.
The file is right: the Hub lists it at 10,946,618 bytes and its SHA-256 equals the pin (`610922cd...`). That shard
holds only metadata, and `check_shards()` takes `g.data_start` (the header rounded up to the alignment) as its least
length, which a tensor-less file never reaches. A shard with no tensors has no tensor data to be short of.

## What changed
- `setup.py`, `check_shards()`: `if have < need:` becomes `if have < need and g.tensors:`.
- `tools/test_shards.py`: `test_tensorless_shard_needs_no_alignment_padding` (a 24-byte GGUF with no tensors passes).
- Left alone: shards with tensors are checked as before (the short-shard and #657 tests are unchanged and pass); the
  pinned SHA-256 checks still cover the tensor-less shard.

## How it was checked
- `python -m unittest tools.test_shards`: the new test fails on `main` 296b59d2 with the "is short" message and passes
  with the change; 14 of 14 pass.
- `tools/test_setup_*.py`, all 40, on `main` and on this branch: every test ends with the same result on both (run on
  macOS, where 15 of them fail on `main` too for PC-only reasons such as the ROCm wheels).
- End to end on a Mac Studio M5 Max 64 GB (macOS 27.0.1): with this change setup finished for UD-Q5_K_XL, and the
  engine serves it (Metal, experts streamed from the SSD).
- Not built: no engine code is touched.

En el sitio

Enlaces a install, modelos, releases.