Issues / #3088
#3088 Per-expert overlay for a native pack, and content hashing for saved sessions
open · @mikhashev · 0 コメント · GitHub で見る
Server & APIMulti-GPUNVIDIA / CUDAModels & quantsWindows
本文
> We run Strata 0.1.45 (`--serve`, Windows) with Qwen3.8-Flash-Next IQ3_XXS as a supervised child of our own agent platform. We want to test whether individual experts carry measurable behaviour — disable or rescale a handful of `(layer, expert)` slots and see what changes. Two things we cannot do ourselves. Everything below is read from the 0.1.45 sources and from our own pack; we have not run a patched model. > > **What we found** > - Our pack has no `experts.bin`, so experts are read from the model's GGUF shard at absolute offsets (`generate.cpp:5408`) and a same-size replacement needs no repack. > - `native_experts.txt` v3 can name a shard per layer, v4 a different one per role (`gate,up,down`) — an overlay at layer and role granularity already exists. > - One expert is **three separate byte ranges**, not one blob: `gate_off`, `up_off`, `down_off` are three stacked tensors at unrelated offsets (layer 47: 46.36, 46.69 and 45.89 GB inside the same 47.04 GB shard). `blob_bytes` is their exact sum — the loader refuses any mismatch with the formats (`expert_layout.cpp:439`), so there is no padding to preserve. > - The load path checks that every `(file, offset)` is the tensor the pack claims, of the pack's type and size (`expert_source.cpp:4602`); nothing hashes the expert bytes at load. > - The session fingerprint is size + first and last MiB of every file, plus its role name (`conversation_file.cpp:938-987`). In our shard no expert byte falls inside either sample: the earliest expert tensor starts at 755 MB and the last one ends 26 MB before the tail sample. A same-size in-place expert edit is therefore invisible to a saved session. > > **Ask 1 — overlay at `(layer, expert)` granularity.** Either extend the shard column, or take a separate file listing `(layer, role, expert) → byte range`, applied over the shard at load, with a printed count of what was replaced. We can patch bytes by hand today; what we cannot do is declare it reproducibly, undo it, and see that it happened. > > **Ask 2 — content hashing for saved sessions, for the in-place case only.** A shard named through the shard column, or an added file, is already noticed — by name, size and samples, and through `native_experts.txt` itself. What is not noticed is an edit inside an existing shard that keeps its size. We would like the fingerprint (or an opt-in flag) to cover an overlay manifest and the ranges it touches, so a session saved across an in-place weight edit is refused rather than resumed. > > **Question:** is the shard column meant as the supported way to substitute weights, or only for splitting the model across files?
関連リンク
インストール・モデル・リリースへの站内リンク。