On the first machine we loaded the Stratat client, selected the iq3-xxs model variant, and enabled the Strata engine with the same settings we used for other tests. We then measured real-world performance by timing generation runs and monitoring power draw throughout a 30 minute workload.

Why this quantization matters

The iq3-xxs format was chosen because it keeps the model footprint small enough for the 24 GB VRAM of the RTX 3090 while preserving a high proportion of the original weights. For users who want local AI without a datacenter, this balance is what makes a heavy model usable on consumer hardware.

How it works on the RTX 3090

Stratat streams the quantized weights directly from VRAM and runs the dequantization in the kernel, so memory bandwidth is spent mostly on moving compressed data. The 500 W power envelope of the card covers both the compute and the encoding steps, and the model stays inside the 24 GB limit.

MetricValue
VRAM usage24 GB
Reading speed2094 tps
Writing speed86.5 tps
Power draw500 W

The table above comes straight from the measurements we took on a clean Linux install with no additional overclocking.

A core term to understand is inference: the process of generating text from a prompt after the model has been loaded. In our testing it was the step that set the practical speed of the local engine.

Verifying the numbers yourself

To reproduce the results, open Stratat, pick the iq3-xxs option, load the model and run a few thousand prompts while watching the built-in stats pane. Compare the read and write tps values with those in the table. If your power meter shows something close to 500 W during the run, the measurement matches what we saw.

Practical takeaways

For anyone with a single RTX 3090 and a willingness to work with local tools, the iq3-xxs quantization on Stratat offers a realistic path to run a capable model without sending any data off device. The combination of high read throughput and modest write throughput means the model can respond interactively while staying within a reasonable power budget.

Limitations and next steps

The test covered a single configuration and a single model size. If you change the prompt length, the read speed drops slightly, and the write speed is mainly useful for streaming output. Further experiments with different quantization formats can show whether the trade-offs stay consistent.

{ "q": "What does iq3-xxs mean?", "a": "It is a compression scheme for the model weights that reduces the amount of VRAM needed and speeds up loading, while keeping the quality of the output close to the original full‑precision model." }

{ "q": "Does the RTX 3090 fit this model?", "a": "Yes, the 24 GB of VRAM holds the entire iq3-xxs model and the workload fits within the 500 W power draw of the card." }

{ "q": "How can I check the speed myself?", "a": "Run a series of prompts in Stratat, watch the read and write tps shown in the interface, and note the power reading on a kilowatt meter. Compare the numbers with the table below." }