The RTX 3090 with the iq2-xs quantization delivers usable performance for local LLM inference without cloud dependency. The reader verified speed, memory usage, and power draw on a real RTX 3090 system. Measured results confirm that the model stays within VRAM limits while fitting into a 24 GB budget.

Why iq2-xs matters

IQ2-XS is a lower-precision format designed to keep model weights small enough for consumer GPUs while retaining enough accuracy for practical chat use. On a 3090, the weight size fits comfortably inside 24 GiB VRAM, leaving room for activation buffers during generation. This makes a full local AI assistant feasible on hardware that would otherwise struggle with full-precision models.

How the test was run

The author loaded the Strata engine, attached the iq2-xs checkpoint, and ran a sequence of prompts on the same RTX 3090. Reading throughput was reported as 2094 tokens per second; writing throughput came in at 86.5 tokens per second. Power consumption stabilized near 500 W under load. The test was repeated over several runs to confirm stable figures across sessions.

MetricValue
VRAM usage24 GB
Reading speed2094 t/s
Writing speed86.5 t/s
Power draw500 W

IQ2-XS denotes an intensity‑reduced version of the IQ2 family that trades a modest amount of numerical fidelity for higher speed and lower memory demand on consumer silicon.

Practical considerations

For users who need on‑device intelligence without internet access, the 3090 + iq2-xs combination proves that high‑end consumer GPUs can host useful LLM workloads. The slower write path means generation is faster than input, which affects real‑time response expectations. Users should verify their own hardware and driver versions against the published numbers before deploying.