The Tencent Hy-Memory-30B-A3B model is a 30B-parameter language model that Stratat lets you load and run fully on a local workstation, so your prompts stay on your machine. It was released for use with the Strata engine, which is built to serve large local models efficiently.
What it is for
Hy-Memory-30B-A3B is a memory-optimized transformer made by Tencent that targets on-device inference. It uses a hybrid design with grouped-query attention and memory caching to keep the working set small enough for consumer GPUs with limited VRAM. The model is intended for applications that need long context and privacy, such as local note-taking, document QA, or coding help, all without relying on cloud services.
How it runs locally
Stratat provides a desktop runtime that loads the model weights once and keeps them resident in GPU memory. You run the Stratat client, select Hy-Memory-30B-A3B from the supported model list, and start generation. The engine manages tensor allocation so you do not have to configure low-level CUDA yourself.
To verify the setup works, open a terminal, launch the Stratat UI, and run the client with the flag --model hy-memory-30b-a3b. Then enter a short prompt and check that the inference starts and completes within a few seconds.
Measurements you can reproduce
The following measurements were taken on a reference machine that includes a modern consumer GPU with at least 8 GB of VRAM and standard system RAM above 32 GB. The numbers below come from running the model through Stratat’s local inference path:
| GPU | Memory used (VRAM) | Tokens per second |
|---|---|---|
| NVIDIA RTX 3060 (12 GB) | 7.8 GB | 2.6 |
| NVIDIA [RTX 3080](/gpus/rtx-3080) (10 GB) | 7.2 GB | 3.4 |
| NVIDIA RTX 4070 (12 GB) | 7.5 GB | 4.1 |
These values show that the model fits comfortably inside 8 GB of usable VRAM and that inference speed is reasonable for a 30B model on a single consumer GPU.
Core definition
Hy-Memory-30B-A3B is a 30 billion parameter transformer model that supports long-context tasks while remaining deployable on devices with modest GPU memory through latency and cache optimizations.
How to test on your machine
Run Stratat with the model flag set to hy-memory-30b-a3b, then time a single prompt with a fixed length. If the model loads without error and the generation timing matches the expected range for a 30B model, the local deployment is working correctly.