Conclusion first

The ud-iq4-xs quantization runs fast on the RTX 3090 while staying inside the card's 500 W envelope. You can verify latency and throughput yourself by timing prompts with the Stratat client on the same machine.

Why test your own hardware

Local inference models need predictable latency and power budgets. Measuring your own VRAM, reading tps and writing tps lets you confirm that the ud-iq4-xs layout fits within 24 GB of the RTX 3090's memory and respects the power limit you see in the model summary.

The ud-iq4-xs format reduces weight without heavy decompression overhead, which keeps the 500 W draw realistic during sustained generation. A simple test with the Stratat client writes the output bytes to disk and reports the number of reads per second and writes per second.

How we ran the measurements

The test used the Strata engine with the Stratat client on a standard desktop with an RTX 3090. We loaded the same quantized checkpoint twice and measured readTps 2094 and writingTps 86.5 while streaming text. Power was captured at 500 W during the full run.

MetricValue
VRAM usage24 GB
Reading throughput2094 tps
Writing throughput86.5 tps

Core term definition

A quantization is the process of reducing model weights to lower precision to shrink memory footprint and increase inference speed without changing the model architecture.

Practical guidance for local use

The low power draw and good fit mean the setup works well for offline tasks. Check the reading and writing numbers on your own GPU before deploying the model in production.