Issues / #1915

#1915 [Bug]: ModelScope: a truncated shard is renamed to its final name — HEAD gives no Content-Length (total=0 in download())

open · @moondrop12138 · 0 comentarios · En GitHub

Setup & installNVIDIA / CUDAModels & quantsWindows

Descripción

**What happened**

Installing with `STRATA_SOURCE=modelscope` on a mainland-China network (Windows), `--family coder --model IQ1_M --context 65536 --vision no`. Step 5, downloading `IQ1_M/Qwen3.8-Flash-Next-GSQ-RCO-IQ1_M-00001-of-00002.gguf` (29,608,446,496 bytes per ModelScope's file API): the stream ended cleanly at 5,466,520,256 bytes after ~7 minutes (~13 MB/s). No `download interrupted (...); retrying in 10 s ...` line appeared — the socket simply reached EOF — and setup then printed:

```
  Qwen3.8-Flash-Next-GSQ-RCO-IQ1_M-00001-of-00002.gguf:  5466.5 MB

  [X]  Qwen3.8-Flash-Next-GSQ-RCO-IQ1_M-00001-of-00002.gguf is 5,466,520,256 bytes, not 29,608,446,496
       delete it and run setup again (the download restarts)
```

Expected: an interrupted download resumes (as it already does when `total` is known), and a short file is never renamed to its final name.

What made this possible: **ModelScope's `/resolve/` HEAD responses carry no `Content-Length`** (verified with curl, with and without an HTTP proxy):

```
$ curl -sI https://www.modelscope.cn/models/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF/resolve/master/IQ1_M/Qwen3.8-Flash-Next-GSQ-RCO-IQ1_M-00001-of-00002.gguf
HTTP/1.1 200 OK
Date: Sun, 11 Oct 2026 02:46:47 GMT
Set-Cookie: acw_tc=...
X-Linked-Etag: e11083ba855e7666b48ea3f2db6a9c3a20c18751a012cc24f948de91b7087fad
...no Content-Length anywhere in the headers
```

So in `download()` (setup.py @ 61b3fb5) `total` stays 0, and with `total == 0` the whole safety net is off:

1. the progress line shows only megabytes — `5466.5 MB` with no denominator — which is the visible tell;
2. the first clean EOF counts as finished: `if not total or have >= total: break` (line 1522). A connection *reset* would raise `OSError` and be retried with `Range`, but a clean FIN after partial data looks exactly like a complete download;
3. the end-of-transfer size check is skipped: `if total and part.stat().st_size != total:` (line 1528);
4. `part.replace(dst)` (line 1531) promotes the 5.4 GB fragment to the shard's real filename.

Only afterwards does `verify_sha256()`'s size check against `ms_meta()` (line 1634) fail. And because the `.part` has already been renamed away, the "delete it and run setup again (the download restarts)" instruction discards the 5.4 GB that a `.part`-aware message could have resumed. (Manual recovery does work: rename the fragment back to `…gguf.part` and rerun — I did exactly that after the fix below, and the download resumed from 5.47 GB, ~17 MB/s on a direct connection.)

This is the same `total == 0` defect as #1797 (GitHub's codeload endpoint), a third trigger host and a different aftermath: there, a truncated archive's `.done` mark sticks and every rerun dies with `BadZipFile`; here, a truncated GGUF shard silently receives its final name and the user is told to restart a 29.6 GB download that could have resumed.

**Suggested fix.** `ms_meta()` already knows the file's published size before the download starts (and `download()` consults it right after, for the hash). Use it as `total` when HEAD has none:

```python
     part = dst.with_name(dst.name + ".part")
-    total = 0
-    for attempt in range(5):
+    meta = ms_meta(*ms) if ms is not None else None
+    total = meta[0] if meta and meta[0] else 0       # ModelScope's HEAD gives no Content-Length: use its published size
+    for attempt in range(5 if not total else 0):
```

and reuse `meta` at the end of the function instead of fetching it again:

```python
     part.replace(dst)
-    meta = ms_meta(*ms) if ms is not None else None
     if meta is not None and meta[1]:
```

With `total` set, the existing retry loop already does the right thing on a premature EOF (it re-requests with `Range: bytes={have}-` and appends), so no other change is needed — the three skipped defences (progress denominator, resume-on-early-EOF, final size check) all come back. A more general variant for any host that strips the length: when `total` is still 0, parse it from the first 206 response's `Content-Range: bytes 0-…/29608446496`.

Nice-to-have: when the final check does fail, the message could offer "keep it as `<name>.part` to resume" instead of "delete it".

**Strata version, GPU, OS**

setup.py @ 61b3fb5 (main, 2026-10-10; ready-made engine 0.1.42) · RTX 5070 Ti 16 GB · Windows 10.0.26200 · Python 3.11.15 · `STRATA_SOURCE=modelscope`, `--family coder --model IQ1_M --context 65536 --vision no`

**Log**

The excerpt above is the whole failure: the progress line without a denominator, then the size check. The run's setup.log contains no `download interrupted` line — the early close was a clean EOF. For comparison, the same file after the suggested fix (resumed, minutes into the rerun):

```
  Qwen3.8-Flash-Next-GSQ-RCO-IQ1_M-00001-of-00002.gguf:   5.97 / 29.61 GB (20%)
```

En el sitio

Enlaces a install, modelos, releases.