The rtx4090-iq3-xxs setup runs the local Strata engine with the IQ3-xxs quantization on a desktop class GPU, delivering 4000 reading tps and 109.3 writing tps while drawing 550 watts under typical load.

Why this configuration works

The RTX 4090’s 24 GB of VRAM matches the memory footprint of the iQ3-xxs model, allowing inference without swapping. Quantization to iQ3-xxs reduces precision while keeping the model size within the 4090’s capacity, so the entire workload fits in GPU memory.

How the measurements were done

Measurements were taken on a single machine with the Strata client installed, using the standard benchmarking flow that Strata exposes. Read speed is measured as tokens per second added to the context during streaming, and write speed is measured as tokens committed per second during generation. Power was read from the GPU’s built‑in power meter during idle, mixed, and peak load phases.

MetricValue
vramsGb24
readingTps4000
writingTps109.3

Core definition

"IQ3-xxs" is a lower‑precision weight format for large language models that trims redundant parameters while preserving the bulk of the learned knowledge, enabling fast inference on limited hardware.

Practical implications

A 550 W draw means each full inference session costs roughly the same energy as running a small office monitor for several hours, making local AI deployment feasible where grid power is abundant but portable power is limited.

Key considerations

Power consumption varies with model size and precision; the iQ3-xxs variant stays within the 4090’s thermal envelope, so users can run long sessions without external cooling upgrades.

Summary

The rtx4090-iq3-xxs combination delivers usable local intelligence with measurable speed and acceptable energy use, giving developers a concrete reference point for quantization testing on high‑end consumer GPUs.