Issues / #2470

#2470 Setup fails GGUF shard validation for Qwen3.8-Flash-Next UD-Q5_K_XL

open · @popken-it · 1 Kommentare · Auf GitHub

Setup & installAMD / HIPNVIDIA / CUDAModels & quantsLinux

Beschreibung

First issue report ever. Working solution provided with chatgpt below. 
I am not technical (ordinairy user), so follow questions will be very hard for me to answer.

# Setup fails GGUF shard validation for Qwen3.8-Flash-Next UD-Q5_K_XL

## Summary

During setup of **Qwen3.8-Flash-Next (Unsloth) UD-Q5_K_XL**, the first GGUF shard is rejected as being 6 bytes too short, even though the downloaded file matches the size and SHA-256 hash declared in `setup.py`.

This appears to be related to the GGUF validation logic in `check_shards()` in `setup.py`.

## Environment

- **OS:** CachyOS Linux
- **CPU:** AMD Ryzen 9 7950X
- **GPU:** AMD Radeon RX 7800 XT, 16 GB VRAM
- **System RAM:** 128 GB
- **Strata version:** v0.1.44.1
- **Model:** Qwen3.8-Flash-Next (Unsloth) UD-Q5_K_XL
- **Model repository:** [[unsloth/Qwen3.8-Flash-Next-GGUF](https://huggingface.co/unsloth/Qwen3.8-Flash-Next-GGUF)](https://huggingface.co/unsloth/Qwen3.8-Flash-Next-GGUF)

## Steps to reproduce

1. Check out Strata v0.1.44.1.
2. Select Qwen3.8-Flash-Next (Unsloth) UD-Q5_K_XL during setup.
3. Run `./setup.sh`.
4. Allow setup to download or detect the existing model shards.

## Expected behavior

Setup should validate the downloaded GGUF shards and continue if their contents and integrity are valid.

## Actual behavior

The original setup reports:

```text
Qwen3.8-Flash-Next-UD-Q5_K_XL-00001-of-00006.gguf is short:
10,946,618 of 10,946,624 bytes (6 missing)
```

The relevant entry in `UNSLOTH_BIG_SHARDS` in `setup.py` declares the following expected size and SHA-256:

```python
"Qwen3.8-Flash-Next-UD-Q5_K_XL-00001-of-00006.gguf": (
    10946618,
    "610922cdb1afe1094e95d7a3e86b12ee33317ebbf7320ac9344e21b6ae4bb69f",
),
```

The downloaded file has exactly the declared size and hash:

```text
File size: 10,946,618 bytes
SHA-256:   610922cdb1afe1094e95d7a3e86b12ee33317ebbf7320ac9344e21b6ae4bb69f
```

Downloading the file again from the Hugging Face repository produced the same size and hash.

## Technical observations

Using `GGUFFile` from `tools/gguf_reader.py`, I inspected the first shard and obtained:

```text
Actual file size:          10,946,618
Calculated minimum size:   10,946,624
Number of tensors:         0
Data start:                 10,946,624
```

The current `check_shards()` implementation calculates the minimum required file size using:

```python
need = g.data_start + max(
    (t.offset + (t.expected_bytes() or 0) for t in g.tensors),
    default=0,
)
```

When the parsed shard contains no tensors, the `default=0` means the calculated minimum size is simply `g.data_start`. In this case, that aligned offset is 6 bytes beyond the end of the file.

The size and hash declared in `setup.py` agree with the downloaded file, but the structural validation rejects it before setup can finish.

## Additional test

After modifying the validation locally to skip the tensor-size check when `g.tensors` is empty, setup proceeded to SHA-256 verification of the model shards:

```text
=== Step 5: downloading Qwen3.8-Flash-Next (Unsloth) UD-Q5_K_XL ===
[ok] Qwen3.8-Flash-Next-UD-Q5_K_XL-00001-of-00006.gguf already downloaded
[ok] Qwen3.8-Flash-Next-UD-Q5_K_XL-00002-of-00006.gguf already downloaded
[ok] Qwen3.8-Flash-Next-UD-Q5_K_XL-00003-of-00006.gguf already downloaded
...
checking Qwen3.8-Flash-Next-UD-Q5_K_XL-00001-of-00006.gguf (SHA-256, 0.0 GB) ...
checking Qwen3.8-Flash-Next-UD-Q5_K_XL-00002-of-00006.gguf (SHA-256, 0.7 GB) ...
checking Qwen3.8-Flash-Next-UD-Q5_K_XL-00003-of-00006.gguf (SHA-256, 54.4 GB) ...
```

**Note:** At the time of writing, the full SHA-256 verification and subsequent model loading have not yet been confirmed to complete successfully.

## Questions

1. Is the first shard intentionally a metadata-only or otherwise special GGUF shard with no tensors?
2. Should `check_shards()` handle an empty tensor list differently, rather than treating the aligned data offset as the minimum file size?
3. Is there an upstream fix or a recommended workaround for this model/version combination?

A robust fix should preserve validation of genuine truncated shards while correctly handling valid GGUF files that contain no tensor payload.

Thank you!

Mehr auf der Site

Links zu Install, Modellen, Releases.