On the first startup the client reports 550 W power and 24 GB VRAM. The measured reading throughput is 4000 reads per second, while writing throughput reaches 109.3 writes per second. The 550 W draw stays within typical domestic limits when the model is kept small.
Why iq3-s fits on the RTX 4090
The iq3-s quantization cuts precision enough to hold the full model inside the 24 GB of video memory. With no external swap, the engine keeps all weights resident and avoids latency spikes during long generations. The model is therefore usable entirely on a single local workstation.
How to run it
Power on the machine, launch the Stratat client, select the RTX 4090, choose the iq3-s variant, and point it at the desired prompt. The engine initializes in seconds and begins streaming results. You can verify the load by watching the power meter stay under 600 W and the VRAM usage hovering near 20–22 GB.
| Metric | Value |
|---|---|
| VRAM used | 24 GB |
| Reading TPS | 4000 |
| Writing TPS | 109.3 |
| Power draw | 550 W |
The core term quantization means a reduced precision format that trades a modest amount of accuracy for much smaller model size and faster inference.
Practical notes
- Keep your room ventilated; the 550 W draw adds heat to the chassis.
- Measure your own batch times with a script that logs timestamps and power.
- If you see stalls, try lowering the context window rather than switching back to the full model.
