On the first test, the RTX 5080 with the IQ2-XS quantization served 3712.5 reading tokens per second while writing 80.7 tokens per second, all while consuming about 510 W of power. The 16 GB of VRAM was sufficient, and the fit assessment came back as Good.
Why a local run matters
Running a quantized model locally eliminates server costs and keeps data private. Using the Strata engine on a single RTX 5080 makes it possible to process locally without a datacenter. Measuring your own benchmarks is essential for knowing whether the device meets your needs.
How the measurement was done
The readingTps of 3712.5 and writingTps of 80.7 were recorded with a 16 GB model loaded in the IQ2-XS format. Power draw was logged at 510 W during continuous inference. You can reproduce the measurements by running the same prompt set on your own RTX 5080 and noting the tps and wattage.
| Metric | Value |
|---|---|
| readingTps | 3712.5 |
| writingTps | 80.7 |
| powerWatts | 510 |
The core term "IQ2-XS quantization" means a low‑precision format that reduces model size while keeping quality. This allows a 16 GB VRAM GPU to hold the weights and run the model locally.
FAQ
- How many tokens can be read per second? About 3712.5 tokens per second on the RTX 5080 with the IQ2-XS quantization.
- Does it fit in memory? Yes, the 16 GB VRAM is sufficient for the chosen quantization.
- What is the power cost? Roughly 510 W while running inference, which at typical French rates translates to about 0.25 €/kWh for energy consumed during a session.
| Device | Quantization | VRAM usage |
|---|---|---|
| RTX 5080 | IQ2-XS | 16 GB |
| Laptop equivalent | IQ2-XS | 16 GB |
| Desktop equivalent | IQ2-XS | 16 GB |
Practical tips
Set the Strata engine to use the IQ2-XS format and confirm the VRAM usage stays under 16 GB. Monitor the power draw with a wattmeter and note when the fan profile changes. If you see stable speeds, the configuration is ready for daily use.