The rtx3090ti-ud-iq4-xs configuration delivers a balanced inference experience on the RTX 3090 Ti, effectively utilizing its 24GB of VRAM to maintain high token throughput while managing thermal and power constraints. This setup is particularly suited for users who prioritize stable, continuous generation over raw maximum speed, as the system maintains a steady power draw of 600W to sustain performance without immediate thermal throttling. By understanding the specific metrics of this quantization, you can better anticipate the behavior of your local Strata engine during extended sessions.

Understanding the Hardware Constraints

The RTX 3090 Ti is a powerful card, but it is also known for its high power consumption. When running the ud-iq4-xs quantization, the system draws up to 600 watts under load. This is a critical factor for users with smaller power supplies or those concerned about electricity costs in the United States, where the average rate is approximately $0.17 per kWh. The 24GB of VRAM is fully utilized to hold the model weights and context, ensuring that the model fits well without swapping to system RAM, which would drastically reduce performance.

Measuring Your Local Performance

To verify these metrics on your own machine, you should monitor the Strata engine's output during a benchmark run. Observe the reading tokens per second (TPS) to gauge how quickly the model processes your prompt, and watch the writing TPS to see the generation speed. You can also use hardware monitoring tools like HWInfo to track the GPU power draw in real-time. If your system consistently hits 600W, ensure your cooling solution is adequate to prevent throttling, which would lower the writing TPS below the expected 93.1 tokens per second.

Performance Breakdown

The following table summarizes the key performance indicators for this specific configuration:

MetricValueImplication
VRAM Usage24 GBFull utilization of GPU memory for model fit
Reading Speed2255.1 TPSFast context processing and prompt analysis
Writing Speed93.1 TPSModerate generation speed for coherent output

Evaluating the Fit

The fit is described as Good, indicating that the model size and the quantization method align well with the hardware capabilities. This means you are getting the best possible performance for the resources available. While the writing speed is not the highest possible, it is sufficient for most interactive tasks. The high reading speed ensures that long context windows are handled efficiently, allowing for complex queries to be processed quickly.

Quantization is the process of reducing the precision of the model's weights to save memory and speed up computation, which is essential for running large language models locally on consumer hardware.