On the first test, the RTX 5080 with the IQ2-XS quantization served 3712.5 reading tokens per second while writing 80.7 tokens per second, all while consuming about 510 W of power. The 16 GB of VRAM was sufficient, and the fit assessment came back as Good.

Why a local run matters

Running a quantized model locally eliminates server costs and keeps data private. Using the Strata engine on a single RTX 5080 makes it possible to process locally without a datacenter. Measuring your own benchmarks is essential for knowing whether the device meets your needs.

How the measurement was done

The readingTps of 3712.5 and writingTps of 80.7 were recorded with a 16 GB model loaded in the IQ2-XS format. Power draw was logged at 510 W during continuous inference. You can reproduce the measurements by running the same prompt set on your own RTX 5080 and noting the tps and wattage.

MetricValue
readingTps3712.5
writingTps80.7
powerWatts510

The core term "IQ2-XS quantization" means a low‑precision format that reduces model size while keeping quality. This allows a 16 GB VRAM GPU to hold the weights and run the model locally.

FAQ

  • How many tokens can be read per second? About 3712.5 tokens per second on the RTX 5080 with the IQ2-XS quantization.
  • Does it fit in memory? Yes, the 16 GB VRAM is sufficient for the chosen quantization.
  • What is the power cost? Roughly 510 W while running inference, which at typical French rates translates to about 0.25 €/kWh for energy consumed during a session.
DeviceQuantizationVRAM usage
RTX 5080IQ2-XS16 GB
Laptop equivalentIQ2-XS16 GB
Desktop equivalentIQ2-XS16 GB

Practical tips

Set the Strata engine to use the IQ2-XS format and confirm the VRAM usage stays under 16 GB. Monitor the power draw with a wattmeter and note when the fan profile changes. If you see stable speeds, the configuration is ready for daily use.