#2783 [Feature] Third-party qwen4exp GGUFs via tools/iq_pack.py — measured report (OrcaRouter Uncensored IQ3_XXS, YaRN to 1M)

本文

[Feature] Third-party qwen4exp GGUFs via tools/iq_pack.py — measured report (OrcaRouter Uncensored IQ3_XXS, YaRN to 1M) and a supported-path question

**Summary.** `tools/iq_pack.py --compat-bf16` repacked a third-party `qwen4exp` GGUF
(`orcarouter/Qwen3.8-Flash-Next-Uncensored-GGUF`, IQ3_XXS, 2 shards, ~85 GB) into a
working native pack with no source changes. The v0.1.44.1 release binary runs it
end-to-end, including YaRN x4 to a 1,048,576-token window. Question at the end:
is there appetite for documenting this repack path, or a curated `SUPPORTED_GGUFS`
entry for third-party repositories? Happy to PR whichever is preferred.

**Hardware and software.**

- RTX 5090 32 GB; Ryzen 9 9950X; 96 GB RAM; NVMe
- Windows 11, driver 581.xx, release binary `strata 0.1.44.1` (source at 296b59d)
- Model: `orcarouter/Qwen3.8-Flash-Next-Uncensored-GGUF` — IQ3_XXS
  (`...-IQ3_XXS-00001-of-00002.gguf`, `-00002-of-00002.gguf`).
  Expert mix: routed gate/up IQ3_XXS, routed down IQ4_NL; PLE table IQ4_NL.
- Draft: official MTP chain (`mtp_fetch.py` → `mtp_pack.py` → `mtp_rt.py`, Q2_0).
  The repo's `MTP-draft.gguf` uses llama.cpp `blk.48.*` names and does NOT feed
  `mtp_pack.py`; the official path worked.

**What was done.**

```
python tools/iq_pack.py --gguf <shard1> --out pack-unc-iq3xxs --compat-bf16
```

Pack: 1,079 tensors indexed, 303 served natively from the GGUFs, 459 dense-side
conversions (96 exact, 363 rounded, max abs err ~0.0144); expert and PLE bytes
untouched. Then:

```
--pack pack-unc-iq3xxs --native <shard1> --ple-gguf <shard1>
--expert-profile data/expert-profile.bin --expert-cache auto --prefill auto
--spec 4 --spec-min-p 0.5 --mtp <mtp rt dir>
--max-context 1048576 --rope-scaling yarn --rope-scale 4
--kv int8 --kv-resident 32768 --vram-reserve-mib 1664 --resident-experts
```

**Note relevant to #2396:** the same files run on the native-pack path without
touching the `compress_ratios` check in `read_geometry` — consistent with that
regression being specific to the `--gguf` loader, not the weights.

**Measured** (single runs per depth; corpus haystack with two synthetic needles at
~30% and ~88% positions; `reasoning_effort: xhigh`; engine-reported timings):

| Stage | Window | YaRN | Prompt tok | Decode tok/s | Draft acc | Needles |
|---|---|---:|---:|---:|---:|---|
| baseline | 262,144 | — | 252.0K | 150.5 | 82% | 2/2 |
| A | 524,288 | x2 | 497.6K | 110.2 | 67% | 2/2 |
| B | 1,048,576 | x4 | 907.1K | 107.7 | 74% | 2/2 |

Across 15 probes (5 per stage): 15/15 clean `stop` finishes, zero repetition flags,
early needle 10/10, late needle 7/10 (misses at all scales — a positional effect,
not a depth cliff). Prompt ingest ~2–2.5K tok/s at depth (~7 min TTFT at 907K).
Over-window requests are rejected, never truncated.

**Not tested:** vision (`strata-vision.exe` + the repo's F16 mmproj not wired),
`--batch`, agentic multi-turn trajectories, quality delta vs the official quants,
and anything below ~119K prompt depth. Needle tests measure retrieval on these
inputs only; single-sample misses are stochastic.

**Ask.** Would you like (a) a docs page documenting `iq_pack.py` repacking for
third-party `qwen4exp` GGUFs (prereqs: `gguf-py` checkout, per-type kernel
coverage, the `--spec`/`--prefill` requirement, MTP fetch), (b) orcarouter quants
in `SUPPORTED_GGUFS` once sha256s are reviewed, or (c) neither — keep it unofficial?
I can PR (a) or the bench-results folder; whichever you prefer.