Issues / #1797
#1797 setup.py get_llama_cpp(): a truncated llama.cpp zip is marked done, then fails with BadZipFile on every rerun
closed · @agnaylor · 1 Kommentare · Auf GitHub
Beschreibung
**Environment**
- Ubuntu 26.04.1, kernel 7.0.0-34, 2× Intel Arc Pro B70 (`xe`)
- compute-runtime 26.31.39395.14, oneAPI DPC++ 2026.1.1
- Native build (no Docker), AOT `-DSTRATA_SYCL_AOT=bmg-g31`
- Strata `fb58e0d` (v0.1.41)
- Seen once, while a parallel ~450 GB model download saturated the link
**Summary.** There are three problems that add up:
1. GitHub's archive endpoint sends no `Content-Length`. On 2026-10-09 a HEAD request to `LLAMA_CPP_ZIP`
(`setup.py:167`) got a 302 with `Content-Length: 0` and then a 200 with no length, so `total == 0` in `download()`
(`setup.py:1338-1340`).
2. With `total == 0`, `download()` takes a stream that ends early as complete. The loop stops at the first clean end
of stream (`setup.py:1385`, `if not total or have >= total: break`), the final size check is skipped
(`setup.py:1391`, `if total and ...`), and the file gets its `.done` mark (`setup.py:1401`).
3. `get_llama_cpp()` (`setup.py:1563-1593`) opens the zip with no validation (`:1572`) and deletes the zip and its
`.done` mark only on success (`:1591-1592`). After a `BadZipFile`, the next run sees `.done` (`setup.py:1317`),
skips the download, and fails again. Recovery means deleting `third_party/llama.cpp-3cf0325.zip{,.done}` by hand.
**Repro (deterministic, without needing a slow link)**
```
cd Strata
head -c 1000000 /dev/urandom > third_party/llama.cpp-3cf0325.zip # stand-in for a truncated archive
touch third_party/llama.cpp-3cf0325.zip.done
rm -rf third_party/llama.cpp
./setup.sh --backend sycl
```
In the wild: run setup while another process saturates the link, so the codeload stream closes early.
**Expected:** setup notices the bad archive, deletes it and its mark, downloads it again (with a few retries), and
only fails with a clear message if it stays bad.
**Actual:** a `zipfile.BadZipFile: File is not a zip file` traceback, and the same traceback on every later run.
**Suggested fix**
- Pin the archive's SHA-256 next to `LLAMA_CPP_COMMIT` (`setup.py:166-167`) and check it with the existing
`verify_sha256()` (`setup.py:1490`) before marking it done. Or, at minimum, have `download()` check for a known
type when `total == 0` (for zips: `zipfile.is_zipfile()` / `ZipFile.testzip()`).
**Related, and the fix already exists one function over:** #424 reported the same sticky `BadZipFile` for the
**engine** archive. 0.1.40 fixed it with `drop_archive()` (`setup.py:1306`), called from `get_prebuilt`'s
`except zipfile.BadZipFile:` (`setup.py:2858`). `get_llama_cpp()` never got that change. Open PR #1126 rewrites
these exact lines (`get_llama_cpp` now calls a new `safe_extract()`) but does **not** add the drop, so it does not
fix this. Coordinate with it rather than conflicting.
- Minimum fix (same as #424): in `get_llama_cpp()`, `except zipfile.BadZipFile: drop_archive(z); raise`. The next
run then downloads again.
- Better, wrap the extraction with a retry:
```python
for attempt in range(3):
download(LLAMA_CPP_ZIP, z, "llama.cpp source")
try:
with zipfile.ZipFile(z) as f:
if f.testzip() is not None:
raise zipfile.BadZipFile("CRC mismatch")
f.extractall(tmp, [m for m in f.namelist() if "/tools/ui/" not in m])
break
except zipfile.BadZipFile as e:
warn(f"llama.cpp source archive is damaged ({e}); downloading it again")
drop_archive(z)
else:
fail("the llama.cpp source archive stays damaged", "check the connection and run setup again")
```
- Separately: `download()` catches `OSError` only (`setup.py:1387`). When a chunked stream is cut off mid-chunk,
`http.client.IncompleteRead` is raised, which is an `HTTPException`, not an `OSError`, so the 30-try retry loop
does not cover it. Add `http.client.HTTPException` to that `except`.
Mehr auf der Site
Links zu Install, Modellen, Releases.