The rtxpro6000-iq3-xxs quantization delivers a high-throughput inference solution on the RTX PRO 6000 Blackwell, leveraging 96GB of VRAM to maintain a good fit for complex local models. With a reading speed of nearly 8,988 tokens per second and a writing speed of 171.1 tokens per second, this setup balances raw processing power with efficient resource management. The system operates at a stable 720 watts, providing a predictable energy profile for sustained computational tasks.
Hardware and VRAM Utilization
The RTX PRO 6000 Blackwell provides a substantial 96GB of VRAM, which is the critical factor for the "Good" fit rating of the rtxpro6000-iq3-xxs quantization. This large memory buffer allows you to load larger context windows without hitting out-of-memory errors. To verify this on your own machine, you can monitor the GPU memory allocation during the initial model load phase. The high reading TPS indicates that the PCIe or NVLink bandwidth is effectively feeding the compute units, ensuring that the GPU is not starved for data during the prompt processing phase.
Inference Speed and Power Efficiency
The writing TPS of 171.1 represents the speed at which the model generates output, a metric that defines the user experience for interactive applications. While reading speed is high, the generation speed is moderate, which is typical for highly compressed quantizations like iq3-xxs. The system draws 720 watts during operation, which translates to a specific cost per hour of use. At a local electricity rate of $0.17 per kWh, you can calculate the running cost by multiplying 0.72 kW by $0.17. This data helps you determine if the hardware acceleration justifies the power draw compared to cloud alternatives.
Measured Performance Metrics
| Metric | Value | Unit |
|---|---|---|
| VRAM Capacity | 96 | GB |
| Reading Speed | 8987.9 | tokens/second |
| Writing Speed | 171.1 | tokens/second |
| Power Consumption | 720 | Watts |
Cost Analysis for Local Deployment
Running this configuration locally eliminates API fees, but the electricity cost is a tangible factor. The 720W draw is consistent with heavy tensor core utilization. You can monitor your actual power usage using system monitoring tools to ensure the 720W figure matches your specific hardware configuration under load. The combination of high VRAM and moderate generation speed makes this setup ideal for batch processing or large context analysis rather than ultra-low latency chat.
The iq3-xxs quantization is a highly compressed format that reduces model size significantly while attempting to preserve logical reasoning capabilities through advanced bit-level optimization.