The Mac Studio M5 Ultra with 96GB of unified memory handles the Coder quantization model with a Good fit, delivering a raw reading throughput of 2284.8 tokens per second. This configuration allows for rapid local inference without relying on cloud APIs, though the system draws significant power during active generation.

Performance Analysis

The primary metric for your local setup is the reading throughput, which sits at 2284.8 tokens per second. This high velocity indicates that the memory bandwidth of the M5 Ultra chip is being utilized effectively to feed the model. You can verify this speed by running a simple local benchmark script that measures the time it takes to process the context window. The writing speed, or generation rate, is measured at 64.8 tokens per second, which is standard for this class of local quantized models.

Power and Cost Considerations

The system consumes 360 watts of power during this workload. In the United States, where the average residential electricity price is approximately $0.17 per kWh, running this setup for an hour costs roughly $0.06. While the per-hour cost seems low, the high wattage draw means you should monitor your local grid load if you are running other heavy tasks simultaneously. You can check your power usage by observing the system monitor or using a hardware power meter plugged into the wall.

Hardware Verification

To ensure your Mac Studio is performing as expected, check the memory allocation in your local client. The 96GB capacity is essential for fitting this model, and the system reports a Good fit, meaning there is no excessive swapping or compression overhead. This stability is critical for long coding sessions where context windows grow large.

Measured Specifications

MetricValueNote
VRAM Capacity96 GBUnified memory pool
Reading Speed2284.8 tokens/secContext processing rate
Writing Speed64.8 tokens/secToken generation rate
Power Draw360 WattsActive load consumption

Technical Definition

Quantization is the process of reducing the precision of the model's weights to lower memory usage and increase inference speed without significantly impacting output quality.