Issues / #2470

#2470 Setup fails GGUF shard validation for Qwen3.8-Flash-Next UD-Q5_K_XL

open · @popken-it · 1 comentarios · En GitHub

Setup & installAMD / HIPNVIDIA / CUDAModels & quantsLinux

Descripción

First issue report ever. Working solution provided with chatgpt below. 
I am not technical (ordinairy user), so follow questions will be very hard for me to answer.

# Setup fails GGUF shard validation for Qwen3.8-Flash-Next UD-Q5_K_XL

## Summary

During setup of **Qwen3.8-Flash-Next (Unsloth) UD-Q5_K_XL**, the first GGUF shard is rejected as being 6 bytes too short, even though the downloaded file matches the size and SHA-256 hash declared in `setup.py`.

This appears to be related to the GGUF validation logic in `check_shards()` in `setup.py`.

## Environment

- **OS:** CachyOS Linux
- **CPU:** AMD Ryzen 9 7950X
- **GPU:** AMD Radeon RX 7800 XT, 16 GB VRAM
- **System RAM:** 128 GB
- **Strata version:** v0.1.44.1
- **Model:** Qwen3.8-Flash-Next (Unsloth) UD-Q5_K_XL
- **Model repository:** [[unsloth/Qwen3.8-Flash-Next-GGUF](https://huggingface.co/unsloth/Qwen3.8-Flash-Next-GGUF)](https://huggingface.co/unsloth/Qwen3.8-Flash-Next-GGUF)

## Steps to reproduce

1. Check out Strata v0.1.44.1.
2. Select Qwen3.8-Flash-Next (Unsloth) UD-Q5_K_XL during setup.
3. Run `./setup.sh`.
4. Allow setup to download or detect the existing model shards.

## Expected behavior

Setup should validate the downloaded GGUF shards and continue if their contents and integrity are valid.

## Actual behavior

The original setup reports:

```text
Qwen3.8-Flash-Next-UD-Q5_K_XL-00001-of-00006.gguf is short:
10,946,618 of 10,946,624 bytes (6 missing)
```

The relevant entry in `UNSLOTH_BIG_SHARDS` in `setup.py` declares the following expected size and SHA-256:

```python
"Qwen3.8-Flash-Next-UD-Q5_K_XL-00001-of-00006.gguf": (
    10946618,
    "610922cdb1afe1094e95d7a3e86b12ee33317ebbf7320ac9344e21b6ae4bb69f",
),
```

The downloaded file has exactly the declared size and hash:

```text
File size: 10,946,618 bytes
SHA-256:   610922cdb1afe1094e95d7a3e86b12ee33317ebbf7320ac9344e21b6ae4bb69f
```

Downloading the file again from the Hugging Face repository produced the same size and hash.

## Technical observations

Using `GGUFFile` from `tools/gguf_reader.py`, I inspected the first shard and obtained:

```text
Actual file size:          10,946,618
Calculated minimum size:   10,946,624
Number of tensors:         0
Data start:                 10,946,624
```

The current `check_shards()` implementation calculates the minimum required file size using:

```python
need = g.data_start + max(
    (t.offset + (t.expected_bytes() or 0) for t in g.tensors),
    default=0,
)
```

When the parsed shard contains no tensors, the `default=0` means the calculated minimum size is simply `g.data_start`. In this case, that aligned offset is 6 bytes beyond the end of the file.

The size and hash declared in `setup.py` agree with the downloaded file, but the structural validation rejects it before setup can finish.

## Additional test

After modifying the validation locally to skip the tensor-size check when `g.tensors` is empty, setup proceeded to SHA-256 verification of the model shards:

```text
=== Step 5: downloading Qwen3.8-Flash-Next (Unsloth) UD-Q5_K_XL ===
[ok] Qwen3.8-Flash-Next-UD-Q5_K_XL-00001-of-00006.gguf already downloaded
[ok] Qwen3.8-Flash-Next-UD-Q5_K_XL-00002-of-00006.gguf already downloaded
[ok] Qwen3.8-Flash-Next-UD-Q5_K_XL-00003-of-00006.gguf already downloaded
...
checking Qwen3.8-Flash-Next-UD-Q5_K_XL-00001-of-00006.gguf (SHA-256, 0.0 GB) ...
checking Qwen3.8-Flash-Next-UD-Q5_K_XL-00002-of-00006.gguf (SHA-256, 0.7 GB) ...
checking Qwen3.8-Flash-Next-UD-Q5_K_XL-00003-of-00006.gguf (SHA-256, 54.4 GB) ...
```

**Note:** At the time of writing, the full SHA-256 verification and subsequent model loading have not yet been confirmed to complete successfully.

## Questions

1. Is the first shard intentionally a metadata-only or otherwise special GGUF shard with no tensors?
2. Should `check_shards()` handle an empty tensor list differently, rather than treating the aligned data offset as the minimum file size?
3. Is there an upstream fix or a recommended workaround for this model/version combination?

A robust fix should preserve validation of genuine truncated shards while correctly handling valid GGUF files that contain no tensor payload.

Thank you!

En el sitio

Enlaces a install, modelos, releases.