Issues / #2470
#2470 Setup fails GGUF shard validation for Qwen3.8-Flash-Next UD-Q5_K_XL
open · @popken-it · 1 comentários · No GitHub
Setup & installAMD / HIPNVIDIA / CUDAModels & quantsLinux
Descrição
First issue report ever. Working solution provided with chatgpt below.
I am not technical (ordinairy user), so follow questions will be very hard for me to answer.
# Setup fails GGUF shard validation for Qwen3.8-Flash-Next UD-Q5_K_XL
## Summary
During setup of **Qwen3.8-Flash-Next (Unsloth) UD-Q5_K_XL**, the first GGUF shard is rejected as being 6 bytes too short, even though the downloaded file matches the size and SHA-256 hash declared in `setup.py`.
This appears to be related to the GGUF validation logic in `check_shards()` in `setup.py`.
## Environment
- **OS:** CachyOS Linux
- **CPU:** AMD Ryzen 9 7950X
- **GPU:** AMD Radeon RX 7800 XT, 16 GB VRAM
- **System RAM:** 128 GB
- **Strata version:** v0.1.44.1
- **Model:** Qwen3.8-Flash-Next (Unsloth) UD-Q5_K_XL
- **Model repository:** [[unsloth/Qwen3.8-Flash-Next-GGUF](https://huggingface.co/unsloth/Qwen3.8-Flash-Next-GGUF)](https://huggingface.co/unsloth/Qwen3.8-Flash-Next-GGUF)
## Steps to reproduce
1. Check out Strata v0.1.44.1.
2. Select Qwen3.8-Flash-Next (Unsloth) UD-Q5_K_XL during setup.
3. Run `./setup.sh`.
4. Allow setup to download or detect the existing model shards.
## Expected behavior
Setup should validate the downloaded GGUF shards and continue if their contents and integrity are valid.
## Actual behavior
The original setup reports:
```text
Qwen3.8-Flash-Next-UD-Q5_K_XL-00001-of-00006.gguf is short:
10,946,618 of 10,946,624 bytes (6 missing)
```
The relevant entry in `UNSLOTH_BIG_SHARDS` in `setup.py` declares the following expected size and SHA-256:
```python
"Qwen3.8-Flash-Next-UD-Q5_K_XL-00001-of-00006.gguf": (
10946618,
"610922cdb1afe1094e95d7a3e86b12ee33317ebbf7320ac9344e21b6ae4bb69f",
),
```
The downloaded file has exactly the declared size and hash:
```text
File size: 10,946,618 bytes
SHA-256: 610922cdb1afe1094e95d7a3e86b12ee33317ebbf7320ac9344e21b6ae4bb69f
```
Downloading the file again from the Hugging Face repository produced the same size and hash.
## Technical observations
Using `GGUFFile` from `tools/gguf_reader.py`, I inspected the first shard and obtained:
```text
Actual file size: 10,946,618
Calculated minimum size: 10,946,624
Number of tensors: 0
Data start: 10,946,624
```
The current `check_shards()` implementation calculates the minimum required file size using:
```python
need = g.data_start + max(
(t.offset + (t.expected_bytes() or 0) for t in g.tensors),
default=0,
)
```
When the parsed shard contains no tensors, the `default=0` means the calculated minimum size is simply `g.data_start`. In this case, that aligned offset is 6 bytes beyond the end of the file.
The size and hash declared in `setup.py` agree with the downloaded file, but the structural validation rejects it before setup can finish.
## Additional test
After modifying the validation locally to skip the tensor-size check when `g.tensors` is empty, setup proceeded to SHA-256 verification of the model shards:
```text
=== Step 5: downloading Qwen3.8-Flash-Next (Unsloth) UD-Q5_K_XL ===
[ok] Qwen3.8-Flash-Next-UD-Q5_K_XL-00001-of-00006.gguf already downloaded
[ok] Qwen3.8-Flash-Next-UD-Q5_K_XL-00002-of-00006.gguf already downloaded
[ok] Qwen3.8-Flash-Next-UD-Q5_K_XL-00003-of-00006.gguf already downloaded
...
checking Qwen3.8-Flash-Next-UD-Q5_K_XL-00001-of-00006.gguf (SHA-256, 0.0 GB) ...
checking Qwen3.8-Flash-Next-UD-Q5_K_XL-00002-of-00006.gguf (SHA-256, 0.7 GB) ...
checking Qwen3.8-Flash-Next-UD-Q5_K_XL-00003-of-00006.gguf (SHA-256, 54.4 GB) ...
```
**Note:** At the time of writing, the full SHA-256 verification and subsequent model loading have not yet been confirmed to complete successfully.
## Questions
1. Is the first shard intentionally a metadata-only or otherwise special GGUF shard with no tensors?
2. Should `check_shards()` handle an empty tensor list differently, rather than treating the aligned data offset as the minimum file size?
3. Is there an upstream fix or a recommended workaround for this model/version combination?
A robust fix should preserve validation of genuine truncated shards while correctly handling valid GGUF files that contain no tensor payload.
Thank you!No site
Links install, modelos, releases.