The AMD Radeon RX 9070 XT paired with the IQ2-XS quantization offers a balanced profile for local inference, achieving a reading throughput of 1652 tokens per second while maintaining a moderate power consumption of 454 watts. This configuration fits well within the 16GB VRAM constraint, providing a stable environment for medium-sized models without excessive energy overhead.
Understanding the Metrics
To verify these results on your own Strata client, you should monitor the real-time telemetry provided by the Strata engine. The reading speed indicates how fast the model can generate tokens, which is critical for a responsive chat interface. Meanwhile, the power draw tells you how much electricity your system is consuming during active inference. You can check your own GPU utilization and temperature to ensure the hardware is not throttling under load.
Power Efficiency and Cost
With a power draw of 454 watts, this setup is energy-intensive. For a user in the United States paying the local average of $0.17 per kWh, running this configuration for one hour will cost approximately $0.077. While this is not negligible, the high token generation speed helps amortize this cost by reducing the time spent in active inference.
Performance Summary
The following table summarizes the measured performance characteristics for this specific hardware and quantization combination:
| Metric | Value | Unit |
|---|---|---|
| VRAM Capacity | 16 | GB |
| Reading Speed | 1652 | tokens/sec |
| Writing Speed | 53.8 | tokens/sec |
| Power Consumption | 454 | watts |
The IQ2-XS quantization is a method that reduces the precision of model weights to save memory, allowing larger models to run on consumer hardware with minimal accuracy loss.
Practical Application
Users should note that while reading speed is high, writing speed is lower, which may affect long-form text generation. However, the fit is rated as good, meaning the 16GB VRAM is utilized efficiently without swapping to system RAM. This makes it a viable option for developers who need quick iterative testing of model outputs.