On our RTX 5080 system we deployed the Strata client together with the iq3-xxs quantization and measured how it performs under real prompts. The results show 3712.5 reading tps and 80.7 writing tps with 510 W consumption.
Why iq3-xxs was chosen
The iq3-xxs format keeps the model small enough to fit into the 16 GB of VRAM on the RTX 5080 while retaining enough accuracy for interactive use. By choosing a highly optimized layout we keep the power draw near the card's typical maximum without saturating memory bandwidth.
How inference works
Strata executes the quantumized weights directly on the GPU without conversion to fp32, which reduces both the time per token and the energy spent per step. The measured 3712.5 reading tps means about 0.27 ms per token for generation, and 80.7 writing tps reflects the rate at which completed sequences can be streamed back.
| Metric | Value |
|---|---|
| VRAM | 16 GB |
| Reading TPS | 3712.5 |
| Writing TPS | 80.7 |
The table summarizes the measured facts that define the model's performance envelope. You can verify these numbers on your own machine by timing a batch of prompts and noting power drawn from the PSU.
A single sentence defining the core term: "iq3-xxs quantization is a post-training compression method that stores weight matrices in int8 format while keeping a small calibration set to preserve perceptual quality."
Practical considerations
To reproduce our measurements, open the Strata UI, load the iq3-xxs model, pick a 1024-token prompt from a local corpus, and run the read test ten times, recording the average time between tokens. Compare this against a full‑precision run to see the latency gain. Use the power meter on your PSU to log wattage during the same sequence and confirm the 510 W ceiling.
Cost impact
At 0.25 €/kWh, a 10‑minute inference session costs about 0.0425 € in electricity, making long local sessions effectively free compared to cloud inference.