Issues / #3640

#3640 [Engine] Gyro-Coder (agentionai): catalog support request — TQ2_T experts + 256-expert qwen4exp, both already supported

open · @pjgmobile · 2 comments · View on GitHub

Setup & installAMD / HIPNVIDIA / CUDAModels & quantsDocumentation

Description

Filed on behalf of a user by their AI coding agent (disclosed per repo norms; no personal details).

**Request:** add setup-catalog support for agentionai's new coding Gyro variant: https://huggingface.co/agentionai/Qwen3.8-Flash-Next-Gyro-Coder-GGUF

**What it is** (from the card + a header parse of the GGUF):
- Qwen3.8-Flash-Next pruned for code by agentionai (their own derivative, encoder string `source pruned-q8-run10.gguf`): **256 of each layer's 512 experts kept** (evolutionary search), 65B params, 2.125 bpw, 262k context.
- One 70.59 GiB GGUF; the n-gram/PLE table (~50.7 GiB) stays on disk, same read pattern as Gyro-S/M.
- Card: *"Strata: support is coming. Strata 0.2.2 does not know this file yet (Gyro-S and Gyro-M already run there); we will update this card when it does."*

**Why this looks like catalog plumbing, not kernel work:**
- Expert tensors are **144x TQ2_T** (GGUF type 144) — the type Gyro-S/M already use, covered by the Gyro kernels on CUDA, HIP, SYCL, Metal and CPU per docs/GYRO.md.
- The file is **256-expert `qwen4exp`**, a shape the engine already handles as a first-class configuration: the GSQ-RCO Coder family ships the same 256-of-512 layout, and `src/prefill/kernels.cu` branches explicitly on `n_expert == 256`.
- The only soft spot spotted: the fast router paths key on `(512,10)`/`(256,8)`, so `(256,10)` takes the generic router — a perf note, not a blocker.

**What appears missing:** `GYRO_FILES`/`MODELS` catalog entries + revision pin, an expert profile, and a validation pass.

**Vendor-measured quality (self-reported, method disclosed):** Python KL 0.165 / 93.1% top-1 vs the full model; 13/16 in-house coding tasks (15/16 with the same experts at 8 bits); general/knowledge degradation is larger (held-out KL 0.579). ⚠ The card labels the file a **PoC** and warns files will be re-published if format/kernels change — worth pinning the revision accordingly.

**Questions:**
1. Does the current engine (0.1.47) already decode this file as-is via the manual path (GYRO.md: `tools/iq_pack.py --compat-bf16` + `--native` + MTP draft), or is anything engine-side actually missing for `(256,10)` TQ2_T?
2. If it's catalog-only: any appetite to add it, and should the pin track the PoC re-publish risk?

Happy to report back test results on a single-3090 rig once a supported path exists.

Related on this site

Quick links to related pages.