#4827 [Engine] use a downloaded GGUF of another length when it is a whole GGUF file
open · @Requiem4Eternity · 0 comments · View on GitHub
Setup & installAMD / HIPNVIDIA / CUDAModels & quantsWindows
Description
Problem and result: a downloaded GGUF that is a whole GGUF file but a few bytes longer than the published one stops setup. A user with `Qwen3.8-Flash-Next-Gyro-S-TQ1_0.gguf` got a file of 58,487,015,904 bytes, 96 bytes (three 32-byte alignment blocks) more than the 58,487,015,808 in `GYRO_FILES`, because it was built with different padding. The file is complete and loads; setup said
```text
[X] Qwen3.8-Flash-Next-Gyro-S-TQ1_0.gguf is 58,487,015,904 bytes, not 58,487,015,808
delete it and run setup again (the download restarts)
```
so the only way forward was to throw away a 58 GB file that works. The same check already had the right answer for `--gguf-dir` (`own=True`): a whole GGUF of another length is used with a warning. After this change a download gets that answer too, and the person sees
```text
[!] Qwen3.8-Flash-Next-Gyro-S-TQ1_0.gguf is 58,487,015,904 bytes; the published file is 58,487,015,808.
It is a complete GGUF file, so Strata uses it as it is; it may be another version or a fine-tune,
so check a few answers before you rely on it
```
Related discussion: #3777 (the report above). This is the small fix described there. The `own=True` branch is unchanged and is not the problem being fixed.
Approach: a second early return, next to the one for `--gguf-dir` and below it, for a downloaded file whose length differs and whose header reads as a whole GGUF:
```python
if have != size:
if own: # --gguf-dir: the person's own file is never deleted by setup
if own_gguf_is_whole(s): # another version or a fine-tune: recommend, never refuse
warn(...)
return
fail(...) # "your file was left as it is"
if own_gguf_is_whole(s): # #3777: a download of another build (96 bytes of padding apart) is used too
warn(...)
return
fail(...) # "delete it and run setup again"
```
The rule is kept on the download path rather than moved up to the top of the function, so that no other caller of `verify_sha256` changes: `download()` also checks the Vulkan SDK tarball, the image model's files and whatever `ready_fetch` fetches, and the vision encoder goes through it too. Those stay on exactly the code they ran before.
The rest is unchanged:
- the file is not deleted, not truncated and not moved: the function only reads its header;
- a file that is not a whole GGUF, a truncated one, and a file whose tensor sizes run past the end still stop setup with the same messages; the `[X]` text for a download is unchanged, so nothing that reads it has to change;
- a download that matches the published length still goes to the SHA-256 check, and a truncated one is refused by `check_shards` before this code runs;
- the `--gguf-dir` branch is untouched, including its `have > 0` guard.
A file of the published length reads the same path it did.
Cost of keeping it here: `own_gguf_is_whole` runs twice for a file from `--gguf-dir` whose header does not read as a whole GGUF, and the warning text appears twice in the source. Both are the price of not reaching the other callers. The repeated text is what the code looked like before, too.
What a person sees afterwards. A downloaded file accepted this way is not marked as finished, because it is not the published file: the warning is printed again on the next run, after one header read, and no SHA-256 is computed for it. The line `... downloaded` that `download()` prints after its check follows the warning, which reads oddly but states what happened.
Validation:
Passed:
`python -m unittest tools.test_setup_ownfiles tools.test_setup_gyro tests.terminal.test_download_integrity tests.terminal.test_own_files`
at a9ca1f3a, Python 3.12.5 on macOS 15 (arm64): `Ran 77 tests ... OK`. The new cases in
`tools/test_setup_ownfiles.py::test_other_lengths` cover a download 96 bytes long (accepted, no longer refused), a file that is not a GGUF at the same length (still refused with the old message, and still on disk afterwards) and an own file that is cut short (still refused).
A whole GGUF 96 bytes longer than its pinned length is used as a download and as `--gguf-dir`; a file that is not a GGUF and a file cut short are refused as both; an own file cut short is refused; no file is deleted. The same five states were run twice each, with the same answer the second time.
Failed:
`python -m unittest tools.test_setup_unsloth`: 43 failures, 1 error. The same set fails on `main` at c62d5e3a without this change (43 failures, 1 error), so it is not part of this branch; they look like tests that need a different environment.
Not run:
No CUDA, HIP or SYCL build, and no engine run: the change is in `setup.py`, not in the engine, and it does not touch anything that compiles. No `--yes` setup against a real published model, and no download of a mismatched file from the network.
Limits: the rule trusts the GGUF header. A file whose tensor directory fits inside its length is used even if the last bytes are padding of a different build; if the data itself is damaged, that shows when the model is loaded, or in the SHA-256 check when the length matches, not here.
---
### Where the real-file evidence comes from
The three questions this change answers ("is a file 96 bytes longer used? is a cutoff file refused? is the model file still intact afterwards?") are about real 58 GB files, and a wrong answer costs an hour and a download to find out. The checks were therefore also run on real files, not only on the small GGUFs the unit tests build: a model that is whole with 96 extra bytes, whole at its own length (accepted, with the warning above), and one that is cut short (refused by `check_shards`) were each fed to `setup.check_shards` and `setup.verify_sha256` themselves, with the published length taken from `GYRO_FILES`.
That drill is not part of this branch and nothing in setup imports it. It is on the branch `drill/setup-size-check`, which is this branch plus `tools/setup_size_check_drill.py`, for a maintainer who wants to see it on their own hardware:
```sh
python tools/setup_size_check_drill.py # 5 cases on GGUFs built here, no model, no network
python tools/setup_size_check_drill.py --file /path/to/model.gguf # the real file, published length read from setup.py
python tools/setup_size_check_drill.py --file /path/to/model.gguf --work-dir D:\temp # copy somewhere else
```
It calls `check_shards` and `verify_sha256` themselves rather than restating their rules, and prints setup's own words for each case. What a case should do is not listed in the script: it comes from `own_gguf_is_whole()` on the same bytes, so the tool reports what setup does instead of defining what it should do.
The model file is only read: one copy is made beside it, every case reshapes only the last few blocks of that copy, and the copy is deleted at the end, including after an interrupt or an error. Nothing is renamed or moved and no byte of the model file is written, so whatever happens it is where it was with the bytes it had. It needs one copy's worth of free space, which `--work-dir` can move to another drive.
Checked on that branch, with the files made read-only first (`chmod 444`) so that any write would have failed: 4 GGUF files (512, 506, 518 and 513 bytes, aligned and not) x 4 case orders = 16 runs, every run leaving the file's SHA-256 unchanged and no copy behind; an interrupt in the middle of a run, with the same result; and the script run on the 58,487,015,904-byte model from #3777 on a Windows machine, the copy beside it on the F: drive: 3 cases, all as expected, and the file still 58,487,015,904 bytes with its original SHA-256 afterwards.Related on this site
Quick links to related pages.