On the first startup the client reports 550 W power and 24 GB VRAM. The measured reading throughput is 4000 reads per second, while writing throughput reaches 109.3 writes per second. The 550 W draw stays within typical domestic limits when the model is kept small.

Why iq3-s fits on the RTX 4090

The iq3-s quantization cuts precision enough to hold the full model inside the 24 GB of video memory. With no external swap, the engine keeps all weights resident and avoids latency spikes during long generations. The model is therefore usable entirely on a single local workstation.

How to run it

Power on the machine, launch the Stratat client, select the RTX 4090, choose the iq3-s variant, and point it at the desired prompt. The engine initializes in seconds and begins streaming results. You can verify the load by watching the power meter stay under 600 W and the VRAM usage hovering near 20–22 GB.

MetricValue
VRAM used24 GB
Reading TPS4000
Writing TPS109.3
Power draw550 W

The core term quantization means a reduced precision format that trades a modest amount of accuracy for much smaller model size and faster inference.

Practical notes

  • Keep your room ventilated; the 550 W draw adds heat to the chassis.
  • Measure your own batch times with a script that logs timestamps and power.
  • If you see stalls, try lowering the context window rather than switching back to the full model.