Issues / #2067

#2067 [Feature]: GLM-5.3-Flash (glm5next) support — the glm53f fork proves it works in Strata; AMD/HIP would make it unique

open · @artfix · 0 comments · View on GitHub

BenchmarksSetup & installAMD / HIPNVIDIA / CUDAModels & quantsLinux

Description

Hi! First: thank you for Strata. Real numbers from my machine first, per the house style:

- **My setup:** RX 7900 XTX 24 GB, Ryzen 7 9700X, 32 GB RAM, Linux (Ubuntu 24.04), low-RAM mode, IQ2_XS.
- **Measured:** 60–70 tok/s decode at 131K context. My daily assistant runs on it — a 125B MoE on a gaming PC at that speed still blows my mind.

**The ask:** GLM-5.3-Flash (glm5next) support in mainline Strata.

I checked the open issues first and couldn't find a GLM feature request — if one exists, I'm happy to move this there.

**Why I think this is realistic and not just a wish:**

- The fork [`lighttransport/Strata`](https://github.com/lighttransport/Strata) (branch `glm53f`, "Strata-GLM53F") already implements a native GGUF backend for GLM-5.3-Flash *inside Strata's codebase*: measured **~29 tok/s decode** (2× Tesla V100 32 GB + 160 GiB RAM, MTP depth 1) and **~7.5 tok/s** (Threadripper 1950X, 160 GB DDR4, RTX 5060 Ti 16 GB), on mixed Q2_K_XL. Fixed layers + batched prefill on GPU, routed experts from RAM, optional GPU MTP drafting.
- GLM-5.3-Flash is ~321B total / 18B active per token: 45 layers (34 Kimi-Delta-Attention linear layers + 11 sparse MLA), 8 of 288 experts + 1 shared expert per token, 1 MTP draft layer, vision encoder, 1M context. Strata already ships GDN (gated delta net) kernels — same gated-delta-rule family as KDA — plus hyper-connection handling and the whole MTP pipeline, so a big part of the hard work already exists in this repo.
- Mainline llama.cpp still has glm5next only in an unmerged PR, and the EXL3 offload projects are CUDA-only. **AMD owners currently have no local path to GLM-5.3-Flash at all.** Strata with its HIP backend is the natural home for one — that combination would be unique.

**Honest caveat from my side:** my box has 32 GB RAM, so GLM on *my* machine would stream experts from the SSD and be much slower than Flash-Next — I know that. But AMD users with 64–128 GB (there are quite a few asking in r/LocalLLaMA — the "Project Maya" thread appeared this week) would get a real GLM runner, and I'd happily help test the low-RAM edge case too.

**What I can offer:** a real RX 7900 XTX test box, benchmarks following COMMUNITY_BENCHMARKS.md (same-day control runs, engine logs), patient bug reports with full logs, and beta testing whenever something is ready to try.

<details>
<summary>📧 Contact (click to reveal email)</summary>

[email protected]

</details>

Related on strata.com

Editorial links to help you install, pick models, or read release notes — not part of the upstream thread.