Conclusion first
The ud-iq4-xs quantization runs fast on the RTX 3090 while staying inside the card's 500 W envelope. You can verify latency and throughput yourself by timing prompts with the Stratat client on the same machine.
Why test your own hardware
Local inference models need predictable latency and power budgets. Measuring your own VRAM, reading tps and writing tps lets you confirm that the ud-iq4-xs layout fits within 24 GB of the RTX 3090's memory and respects the power limit you see in the model summary.
The ud-iq4-xs format reduces weight without heavy decompression overhead, which keeps the 500 W draw realistic during sustained generation. A simple test with the Stratat client writes the output bytes to disk and reports the number of reads per second and writes per second.
How we ran the measurements
The test used the Strata engine with the Stratat client on a standard desktop with an RTX 3090. We loaded the same quantized checkpoint twice and measured readTps 2094 and writingTps 86.5 while streaming text. Power was captured at 500 W during the full run.
| Metric | Value |
|---|---|
| VRAM usage | 24 GB |
| Reading throughput | 2094 tps |
| Writing throughput | 86.5 tps |
Core term definition
A quantization is the process of reducing model weights to lower precision to shrink memory footprint and increase inference speed without changing the model architecture.
Practical guidance for local use
The low power draw and good fit mean the setup works well for offline tasks. Check the reading and writing numbers on your own GPU before deploying the model in production.