The RTX 5090 paired with the iq2-xs quantization delivers a highly efficient balance for local inference, fitting comfortably within your 32GB VRAM limit while maintaining strong write speeds. This configuration allows you to run complex models without exceeding hardware constraints, making it a solid choice for users prioritizing stability and moderate throughput over raw maximum speed.
Understanding the Hardware Fit
When you evaluate the RTX 5090 for this specific quantization, the most critical factor is the memory footprint. The measured fit is rated as "Good," which indicates that the model weights and necessary context buffers will reside entirely within the GPU's 32GB VRAM. This eliminates the bottleneck of swapping data to system RAM, ensuring that your inference process remains stable and predictable. You can verify this fit on your own machine by monitoring VRAM usage in your Strata client during the loading phase; if usage stays below the 32GB ceiling, you are safe from out-of-memory errors.
Performance Metrics and Throughput
The performance metrics reveal a system optimized for rapid token generation rather than pure raw read speeds. With a reading throughput of 5803 tokens per second, the system processes input data exceptionally fast. However, the writing speed settles at 168.4 tokens per second. This discrepancy is typical for lower-bit quantizations where the decoding process is the limiting factor. For your local setup, this means you can feed large prompts quickly, but the response generation will be steady rather than instantaneous. To check these numbers, run a standard benchmark script in your Strata client and observe the tokens per second output during a sustained generation task.
Power Consumption and Operating Costs
Running the RTX 5090 at this configuration demands significant energy, with a measured power draw of 700 watts. This high wattage reflects the card's capacity to handle the computational load of the iq2-xs quantization efficiently, but it comes with a tangible electricity cost. To calculate your actual expense, you must apply the local electricity rate of $0.17 per kWh. For example, running this setup for one hour consumes 0.7 kWh, costing approximately $0.12. While this may seem small per session, continuous operation adds up quickly. You can monitor your real-time power draw using hardware monitoring tools to ensure your power supply unit (PSU) can handle the 700-watt peak without throttling.
Key Performance Summary
| Metric | Value | Significance |
|---|---|---|
| VRAM Usage | 32 GB | Determines if the model fits entirely on the GPU |
| Reading Speed | 5803 tps | Indicates how fast input data is processed |
| Writing Speed | 168.4 tps | Indicates the rate of text generation |
| Power Draw | 700 W | Determines electricity cost and cooling requirements |
The iq2-xs quantization is a method that compresses the neural network weights to four bits, significantly reducing memory usage while attempting to preserve the model's reasoning capabilities through advanced compression techniques.