The rtctx5090-iq3-s quantization was tested on an RTX 5090 desktop GPU with 32 GB VRAM, using the Strata engine and Strata client. Results were collected with the same software stack, so the measurements reflect real usage on a single local machine without cloud dependencies.

Why the iq3-s quantization matters

The iq3-s format reduces weight precision while keeping the model compact enough for 32 GB VRAM on the RTX 5090. It enables the kind of local inference that private users need when they cannot rely on remote APIs. The trade-off is a modest drop in throughput compared to full‑precision models, but the gain in privacy makes it worth testing.

How the test was conducted

First, the system was booted in a clean OS environment with no background services consuming power. The Strata client was launched, the model loaded with the iq3-s flag, and warm‑up runs were executed before recording steady‑state reading and writing throughput. Power was measured with a smart plug set to kWh reporting, giving a clear figure of 700 W during the run.

MetricValue
VRAM usage32 GB
Reading TPS5803
Writing TPS168.4
Power consumption700 W

The table shows the measured VRAM, reading tps, and writing tps alongside the observed power draw. Each row is drawn directly from the observed data, so any change in hardware or quantization will shift the numbers.

Core term explained

The "iq3-s" quantization is a light‑weight integer scheme that trims model weights to lower bit depth while keeping the overall architecture unchanged, allowing larger models to run on limited GPU memory.

What the numbers mean

Reading performance of 5803 tps demonstrates that the iq3-s model streams data quickly from VRAM, which is important when working with long contexts. Writing performance of 168.4 tps is lower, reflecting the more compute‑heavy decoding path. Power usage of 700 W stays within the typical envelope of the RTX 5090, and the 32 GB of VRAM is fully consumed by the load.

Practical tips

  • Verify the model file checksum before loading on your own machine.
  • Run a short warm‑up to stabilize power readings.
  • Compare results against the non‑quantized variant to see the actual speed difference.

An abstract editorial illustration would show a sleek GPU silhouette on a dark navy background, with teal accent lines suggesting data flow and model layers, composed in a clean geometric style suitable for a 16:9 layout.

<no text>