问题 / #3269

#3269 [Other] 0.2.0 Linux control package: tools/iq_pack.py can't find gguf-py; the "run setup.py" hint doesn't fix it in a control-mode install

open · @talisp · 2 评论 · 去 GitHub 看

BenchmarksSetup & installAMD / HIPNVIDIA / CUDAModels & quantsLinux

说明

### Area

[Other]  (anything else, including setup and installation)

### Desktop app, Terminal or source?

Other

The Linux control package (`strata-control-0.1.45.3-linux-x86_64.tar.gz`), unpacked, with its bundled packer run by hand. No desktop app.

### What happened

I ran the bundled packer by hand to make packs for six Unsloth Qwen3.8-Flash-Next GGUFs (UD-Q2_K_XL to Q8_0). It stops after listing the shards with:

```
llama.cpp's gguf-py was not found: run setup.py (it clones llama.cpp into third_party/), or set STRATA_GGUF_PY
```

**Cause (from reading the code)**

- The package has no `third_party/llama.cpp/gguf-py`.
- `tools/_paths.py` looks only in `$STRATA_GGUF_PY`, `<control>/third_party/llama.cpp/gguf-py` and the dev tree's `.ref/llama.cpp/gguf-py`.
- In control mode (`STRATA_CONTROL_INSTALL=1`), `setup.py` writes `third_party/` to `home()` (`STRATA_INSTALL_DIR`), not to `ROOT`. So the hint "run setup.py" doesn't help a tool run by hand: the tool never looks in `home()/third_party`.
- `setup.py` itself is fine: it sets `STRATA_GGUF_PY` when it calls the tools (e.g. at step 6). Only a tool run directly from the package hits this.

**Workaround (works)**

Point `STRATA_GGUF_PY` at any llama.cpp `gguf-py` (I used the one from a 0.1.45 source install):

```
STRATA_GGUF_PY=<llama.cpp>/gguf-py python/bin/python3 tools/iq_pack.py --gguf ... --out ... --compat-bf16
```

With that, the UD-Q2_K_XL pack takes ~25 s. The engine starts in 39 s and serves normally on the 0.2.0 engine: ~125 tok/s decode at 8K and ~110 tok/s at 128K, with MTP on.

**Suggestion**

Any one of these would fix it:
- ship `gguf-py` in the control package;
- have `_paths.py` also check `$STRATA_INSTALL_DIR/third_party/llama.cpp/gguf-py`;
- change the error message to name `STRATA_GGUF_PY` as the fix for packages.

### Smallest steps or command that reproduce it

1. Unpack `strata-control-0.1.45.3-linux-x86_64.tar.gz` (SHA-256 OK against SHA256SUMS.txt).
2. Run:

```
cd <control> && python/bin/python3 tools/iq_pack.py --gguf <model>-00001-of-0000N.gguf --out <packs>/<m> --compat-bf16
```

### Reproduction status

Reproduced locally

### Strata version

0.2.0-beta (control and engine 0.1.45.3, Linux x86_64, engine package cuda12)

### Graphics card, memory and system

NVIDIA L40S-48Q (vGPU, sm_89, 48 GB), driver 550.127.05; 251 GiB RAM; Xeon Platinum 8580 (32 vCPU); Ubuntu 22.04

### Model, size and settings

unsloth/Qwen3.8-Flash-Next-GGUF UD-Q2_K_XL (73.5 GiB, revision 766911a), `--compat-bf16`

### Logs

```
llama.cpp's gguf-py was not found: run setup.py (it clones llama.cpp into third_party/), or set STRATA_GGUF_PY
```

本站相关内容

相关页面的快捷入口。