Niko Veit（@coldniko） / 2026-10-08
Project Maya deep dive: running GLM-5.3-Flash (321B MoE) on your own NVIDIA GPUs, built on Strata
Site analysis · Project Maya (mw00/project-maya) is a Strata-derived engine rewrite that targets zai-org/GLM-5.3-Flash, a 321B-parameter MoE (about 18B active per token) with up to 1M-token context. How the port works, per the project's own study (docs/GLM5-FLASH.md, written against engine 0.1.27): roughly two thirds of Strata's machinery carries over unchanged because GLM-5.3-Flash is the same kind of model as Qwen3.8-Flash-Next — a 3:1 linear-attention/sparse-attention hybrid MoE with hyper-connections and an MTP layer. The reworked third is the mixer (KDA instead of GDN), the full-attention mixer (MLA instead of plain QKV), the router scoring, and the model packaging. Memory is the whole game: the engine keeps the most-used experts in VRAM, the next tier in pinned RAM, and the rest on NVMe SSD, moving them as the conversation needs — nothing leaves the machine. On 2x Tesla V100 32GB the Maya-S quant decodes at up to 40 tokens/s with prefill up to 560 tokens/s; a single V100 still reaches ~19 tokens/s. The quants are calibrated, not squeezed: Maya-S (96.5 GB) keeps 97.9% of the FP8 model's zero-shot accuracy and matches its next token 83% of the time; Maya-M (116 GB) adds IQ3_XXS down projections and IQ3_S in the most sensitive layers for 23% lower KL divergence and 86% token agreement, calibrated toward tool calls and front-end code. Multi-GPU is pipeline (layer) parallelism, not tensor parallelism — a token crosses cards once per verify window, no NVLink needed. Recent v1.0.x releases added a comparable --bench mode, Turing long-prompt fixes, and Maya-M. Full analysis: https://github.com/mw00/project-maya