The rx907070xt_16gb-ud-iq4-xs quantization is a solid fit for the AMD Radeon RX 9070 XT, balancing high throughput with manageable power consumption. You get 1,652 tokens per second in reading speed while drawing 454 watts, which is efficient for the performance delivered. This setup allows you to run large language models locally without exceeding your 16GB VRAM limit.

Why This Configuration Works

The primary advantage of this setup is the fit. The 16GB of VRAM on the RX 9070 XT is just enough to hold this specific model configuration comfortably. If you try to load a larger model, you will run out of memory. The ud-iq4-xs quantization reduces the model size enough to fit while keeping the quality high enough for most tasks.

How to Measure Your Results

You can verify these numbers on your own machine using the Strata client. First, ensure your AMD drivers are up to date. Then, start the Strata engine and load the model. Watch the dashboard for the reading and writing speeds. You should see reading speeds near 1,652 tokens per second. The power draw will stabilize around 454 watts. This helps you understand your local electricity costs.

Performance Metrics

MetricValueNotes
VRAM16 GBTotal available memory on the GPU
Reading Speed1652 TPSTokens per second for generation
Writing Speed53.8 TPSTokens per second for processing
Power Draw454 WAverage power consumption under load
FitGoodModel fits within VRAM constraints

Understanding the Fit

The term "fit" describes how well a model fits into the available VRAM without swapping to system RAM, which would slow down performance significantly.

Cost Considerations

Running this model costs about $0.77 per hour at $0.17/kWh. This is a reasonable cost for local inference. You save on API fees but pay with electricity. The 454-watt draw is significant, so ensure your power supply can handle it.