Issues / #3269
#3269 [Other] 0.2.0 Linux control package: tools/iq_pack.py can't find gguf-py; the "run setup.py" hint doesn't fix it in a control-mode install
open · @talisp · 2 コメント · GitHub で見る
BenchmarksSetup & installAMD / HIPNVIDIA / CUDAModels & quantsLinux
本文
### Area [Other] (anything else, including setup and installation) ### Desktop app, Terminal or source? Other The Linux control package (`strata-control-0.1.45.3-linux-x86_64.tar.gz`), unpacked, with its bundled packer run by hand. No desktop app. ### What happened I ran the bundled packer by hand to make packs for six Unsloth Qwen3.8-Flash-Next GGUFs (UD-Q2_K_XL to Q8_0). It stops after listing the shards with: ``` llama.cpp's gguf-py was not found: run setup.py (it clones llama.cpp into third_party/), or set STRATA_GGUF_PY ``` **Cause (from reading the code)** - The package has no `third_party/llama.cpp/gguf-py`. - `tools/_paths.py` looks only in `$STRATA_GGUF_PY`, `<control>/third_party/llama.cpp/gguf-py` and the dev tree's `.ref/llama.cpp/gguf-py`. - In control mode (`STRATA_CONTROL_INSTALL=1`), `setup.py` writes `third_party/` to `home()` (`STRATA_INSTALL_DIR`), not to `ROOT`. So the hint "run setup.py" doesn't help a tool run by hand: the tool never looks in `home()/third_party`. - `setup.py` itself is fine: it sets `STRATA_GGUF_PY` when it calls the tools (e.g. at step 6). Only a tool run directly from the package hits this. **Workaround (works)** Point `STRATA_GGUF_PY` at any llama.cpp `gguf-py` (I used the one from a 0.1.45 source install): ``` STRATA_GGUF_PY=<llama.cpp>/gguf-py python/bin/python3 tools/iq_pack.py --gguf ... --out ... --compat-bf16 ``` With that, the UD-Q2_K_XL pack takes ~25 s. The engine starts in 39 s and serves normally on the 0.2.0 engine: ~125 tok/s decode at 8K and ~110 tok/s at 128K, with MTP on. **Suggestion** Any one of these would fix it: - ship `gguf-py` in the control package; - have `_paths.py` also check `$STRATA_INSTALL_DIR/third_party/llama.cpp/gguf-py`; - change the error message to name `STRATA_GGUF_PY` as the fix for packages. ### Smallest steps or command that reproduce it 1. Unpack `strata-control-0.1.45.3-linux-x86_64.tar.gz` (SHA-256 OK against SHA256SUMS.txt). 2. Run: ``` cd <control> && python/bin/python3 tools/iq_pack.py --gguf <model>-00001-of-0000N.gguf --out <packs>/<m> --compat-bf16 ``` ### Reproduction status Reproduced locally ### Strata version 0.2.0-beta (control and engine 0.1.45.3, Linux x86_64, engine package cuda12) ### Graphics card, memory and system NVIDIA L40S-48Q (vGPU, sm_89, 48 GB), driver 550.127.05; 251 GiB RAM; Xeon Platinum 8580 (32 vCPU); Ubuntu 22.04 ### Model, size and settings unsloth/Qwen3.8-Flash-Next-GGUF UD-Q2_K_XL (73.5 GiB, revision 766911a), `--compat-bf16` ### Logs ``` llama.cpp's gguf-py was not found: run setup.py (it clones llama.cpp into third_party/), or set STRATA_GGUF_PY ```
関連リンク
インストール・モデル・リリースへの站内リンク。