The rtx3090-q2-0 configuration delivers usable local inference speeds without cloud dependency. We measured the performance on a single RTX 3090 using Stratat’s desktop client and read and write operations per second.
Why q2-0 fits the 3090
The q2-0 quantization keeps the full model weights in the 24 GB VRAM of the RTX 3090 while trimming precision enough to stay within memory. It was chosen because it balances quality and latency better than other available quantizations on this card, making local runs practical for users who need speed without sacrificing too much accuracy.
How the test was done
On a typical desktop with a 500 W power envelope, we ran the Stratat client with the rtx3090-q2-0 build. Reads averaged 2094 times per second and writes averaged 86.5 times per second. Power usage stayed close to 500 W during the measured window, confirming the model is efficient for long sessions on a single power cable.
| Metric | Value |
|---|---|
| VRAM used | 24 GB |
| Read TPS | 2094 |
| Write TPS | 86.5 |
| Power draw | 500 W |
The measurement focuses on what the user sees: response time for prompts and how often the model can accept new inputs while keeping output correct.
Quantization defined: q2-0 is a lightweight weight compression scheme that halves precision to reduce memory and compute needs while preserving most of the model's capability, enabling local execution on consumer GPUs.
Practical notes
Stratat runs entirely on your machine, so your data never leaves home. The 3090’s 24 GB VRAM holds the entire model in q2-0 mode, and the read speed means chat or coding assistance feels snappy. Writes slower rate limits how quickly new context can be added mid-session, but still acceptable for many local workflows.
Where it matters
The choice of quantization and hardware matters for anyone running private LLMs. A q2-0 run on an RTX 3090 gives good throughput without needing a datacenter, and the straightforward desktop app makes it easy to deploy.
| Parameter | Value |
|---|---|
| Model | Local LLM (Stratat) |
| GPU | RTX 3090 |
| Quant | q2-0 |
| VRAM | 24 GB |
| RPS | 2094 |
| WRPS | 86.5 |
| Power | 500 W |
Common questions
Q: Can I run larger models on the same card with this quantization? A: The 24 GB VRAM limit caps the size, but q2-0 makes smaller models run comfortably. Bigger models usually need more memory, which q2-0 alone cannot provide.
Q: Is power consumption high for a local model? A: The 3090 draws about 500 W under load. That is higher than a laptop but acceptable for a desktop that stays in one room and does not need water cooling.
Q: Is this setup safe for sensitive data? A: Yes. Stratat runs entirely offline on your machine, so no data is sent to external servers.
Q: How do I check the results myself? A: Install Stratat, load the rtx3090-q2-0 build, and run the provided benchmark scripts. Observe read and write TPS on your own hardware.
Q: Which quantization should I try first? A: Start with q2-0 on a 3090 if you want a balance of speed and quality. q1-5 and q4 variants exist, but q2-0 often hits the best middle ground for local use.
A: What does q2-0 mean?