The RTX 4090 delivers exceptional performance with the UD-IQ4-XS quantization, achieving 4000 tokens per second in reading and 109.3 tokens per second in writing while maintaining a good fit within its 24GB VRAM limit. This configuration represents a highly efficient balance between model capacity and inference speed for local desktop environments.

Understanding the Hardware Fit

The primary constraint for local model execution is Video RAM (VRAM). With 24GB available, the UD-IQ4-XS quantization fits comfortably, described as a "Good" fit. This ensures that the model weights reside entirely in the GPU memory, eliminating the severe performance penalties associated with system RAM swapping. Users can verify this by monitoring memory usage during the initial load phase of the Strata engine.

Token Throughput and Efficiency

Performance is measured in tokens per second (TPs). The reading speed of 4000 Tps allows for near-instantaneous context processing, making the interface feel responsive. The writing speed of 109.3 Tps provides a smooth, readable generation rate for complex tasks. These figures are derived directly from the hardware capabilities of the RTX 4090 under this specific quantization.

Power Consumption and Cost

The system draws 550 watts during active inference. At the US average electricity rate of $0.17 per kWh, running the GPU at full load for one hour costs approximately $0.09. While this is a significant power draw, the high throughput justifies the energy cost for intensive tasks. Users can monitor real-time power draw using system monitoring tools to track efficiency.

MetricValueUnit
VRAM Capacity24GB
Reading Speed4000Tps
Writing Speed109.3Tps
Power Draw550Watts

The Strata engine optimizes memory allocation to ensure the model remains stable and fast. This stability is crucial for long-running sessions.

Core Definition

The UD-IQ4-XS quantization is a specific compression method that reduces model size to fit within 24GB VRAM while preserving high inference speed. This definition highlights the balance between size and performance.