#6013 [Other] --ple-gguf + Unsloth engram table: the manual swap that works today (re-stamp the header to strata-ple; verified UD-Q4_K_XL, engine 0.1.59.1)
closed · @artfix · 1 comentarios · En GitHub
Setup & installAMD / HIPNVIDIA / CUDAModels & quantsDocumentationLinux
Descripción
### Area
Other
### What is this?
A **verified working recipe** for running an Unsloth engram n-gram table (e.g. the Q8_0 table) on Strata via `--ple-gguf` with a native multi-shard Unsloth set. Relevant to #5248 — this is the manual path that works today, plus one small engine idea that would make it automatic. Documentation-worthy either way: nobody outside llama.cpp seems to know how to attach an engram table to Strata, and the failure mode is cryptic.
### Setup where this was verified
- Engine `0.1.59.1` (Linux, HIP gfx1100), Strata setup with the pinned Unsloth `UD-Q4_K_XL` 4-shard native set (pack + `--native` shard 1), `--ple-io` unset (default `direct`, table on NVMe).
- Old table: bundled IQ4_NL (found by name in shard 2). Target: `engram/Qwen3.8-Flash-Next-engram-Q8_0.gguf_file` — the same 54,400,261,312-byte tensor as UD-Q6_K_XL's shard `00003-of-00006`.
### What does NOT work (both fail at boot, cleanly)
1. `--ple-gguf <UD-Q6 shard 00003-of-00006>` (or the engram file as published):
```
strata generate: native dense: additional shard must match the architecture-validated first shard's split metadata
```
2. Same file after re-stamping `split.no`/`split.count` with Unsloth's `engram/set_split.py`: same error — the check additionally requires `split.tensors.count` to match, and a complete native set has no free shard slot to claim anyway (`native dense: duplicate split shard number`).
### Why: the contract in your own source
`src/program/generate.cpp` (the `--ple-gguf` wiring):
```cpp
// a PLE-only table (tools/ple_fp8_pack.py: architecture strata-ple) holds no projections
if (const strata::MetaValue* v = pg.get("general.architecture")) ple_only = v->s == "strata-ple";
if (!ple_only && ...) o.native_dense_gguf.push_back(o.ple_gguf); // <- everything else becomes a "dense shard"
```
Unsloth's engram files (and the table shards) have **no `general.architecture` key at all** — only `split.no` / `split.count` / `split.tensors.count` — so `ple_only` is always false and the table is routed into `NativeDense::load`, which is right to refuse it.
### The fix: re-stamp the table header to `strata-ple` (192 bytes, data untouched)
The whole change is in the GGUF header: set `general.architecture = "strata-ple"`, drop the `split.*` keys, keep the tensor-info bytes, and pad a filler KV so the tensor-info end stays in the same 32-byte alignment window — `data_start` keeps its exact value and tensor offsets are data_start-relative, so the 54 GB payload is never touched. `ple_format_for_type()` then picks Q8_0 from the tensor type itself (`kPleFormats` even names `Unsloth's UD-Q6_K_XL` in its comment), so no `strata.ple.format` key is needed.
```python
#!/usr/bin/env python3
"""Re-stamp a standalone engram table GGUF so Strata --ple-gguf reads it as a PLE-only table.
Dry run by default; --apply rewrites the header in place. Tensor data is never touched."""
import struct, sys
path = sys.argv[1]; apply = "--apply" in sys.argv
f = open(path, "r+b"); h = f.read(1 << 20)
assert h[:4] == b"GGUF"; ver, nt, nkv = struct.unpack("<IQQ", h[4:24])
def rds(p): return p + 8 + struct.unpack("<Q", h[p:p+8])[0]
ES = {0:1,1:1,2:2,3:2,4:4,5:4,6:4,7:1,10:8,11:8,12:8}
def skipv(p, t):
if t == 8: return rds(p)
if t == 9:
et = struct.unpack("<I", h[p:p+4])[0]; c = struct.unpack("<Q", h[p+4:p+12])[0]; p += 12
for _ in range(c): p = skipv(p, et)
return p
return p + ES[t]
p, keep, nkeep = 24, bytearray(), 0
for _ in range(nkv):
s = struct.unpack("<Q", h[p:p+8])[0]; k = h[p+8:p+8+s].decode("utf-8","replace"); st = p
p = rds(p); t = struct.unpack("<I", h[p:p+4])[0]; p += 4; p = skipv(p, t)
if k != "general.architecture" and not k.startswith("split."): keep += h[st:p]; nkeep += 1
kv_end = p
for _ in range(nt):
p = rds(p); nd = struct.unpack("<I", h[p:p+4])[0]; p += 4 + 8*nd + 4 + 8
ti_len = p - kv_end; ds = (p + 31) // 32 * 32
def gs(s):
b = s.encode(); return struct.pack("<Q", len(b)) + b
arch = gs("general.architecture") + struct.pack("<I", 8) + gs("strata-ple")
start = 24 + len(arch) + len(keep) + ti_len + 30
pad = ((ds - start) % 32 + 32) % 32
assert start + pad <= ds, "header would not fit the alignment window - data would have to move"
new = h[:16] + struct.pack("<Q", nkeep + 2) + arch + bytes(keep) \
+ gs("strata.pad") + struct.pack("<I", 8) + gs("x" * pad) + h[kv_end:kv_end+ti_len]
new += b"\x00" * (ds - len(new))
print(f"n_kv {nkv}->{nkeep+2}, tensor-info end {p}->{start+pad}, data_start stays {ds}")
if apply:
f.seek(0); f.write(new); print("header rewritten:", path)
```
Usage (any UD quant's native set, same table):
```bash
python3 restamp.py Qwen3.8-Flash-Next-engram-Q8_0.gguf_file # dry run
python3 restamp.py Qwen3.8-Flash-Next-engram-Q8_0.gguf_file --apply
# config args: "--ple-gguf", "/path/to/the/stamped/file" (--ple-io left at the direct default)
```
### Verified result (startup log, same machine, same config, before/after)
```
before: strata generate: PLE on, table 320001536 rows of .../Qwen3.8-Flash-Next-UD-Q4_K_XL-00002-of-00004.gguf
after: strata generate: PLE direct read probe: 106298 rows/s (fine)
after: strata generate: PLE on, table 320001536 rows of .../ple-q8/Qwen3.8-Flash-Next-UD-Q6_K_XL-00003-of-00006.gguf
```
Decode speed unchanged (106K rows/s probe on Q8_0's 170 B rows vs ~108K on IQ4_NL's 90 B rows — one NVMe read per row either way). Table error vs BF16 drops from RMSE 0.0757 (IQ4_NL) to 0.0053 (Q8_0) per Unsloth's engram README, for ~26 GB more disk and zero speed.
### One engine idea (optional, would replace the whole hack)
`generate.cpp` already auto-finds the PLE shard by tensor name (`model.find("per_layer_token_embd.weight", &at)`). The same one-liner could apply to the `--ple-gguf` file: if it holds exactly that one tensor whose type is in `kPleFormats`, treat it as table-only regardless of `general.architecture`. Then any Unsloth engram / table shard loads with no re-stamping at all, and `strata setup` could offer `--ple-table q8_0|bf16|q5_1` as #5248 proposes. (Or, at minimum: document the contract — a `--ple-gguf` section in DETAILS.md/QUANTS.md saying "external tables must be architecture=strata-ple or a shard of the set", since the current error reads like a corrupt download.)
Not attaching the stamped file itself (54 GB), but the recipe is complete without it. Happy to test any engine branch that implements the auto-detect.
En el sitio
Enlaces a install, modelos, releases.