What IQ3-S means
IQ3-S is a weight-only quantization that keeps most weights in 3-bit form while using higher precision for activations, reducing memory footprint and increasing speed. For the RTX 3090 it allows the Strata engine to keep the model in VRAM without a swap.
How to measure on your own machine
The read speed of 2094 tps was obtained by loading the model checkpoint and repeatedly calling the load function in a script that counts tokens produced per second. The write speed of 86.5 tps came from measuring how many generated tokens were written back to disk over a fixed time window. VRAM usage stayed within 24 GB and power draw settled near 500 W under the same workload.
Practical setup
The Stratat client on a Windows or Linux desktop loads the IQ3-S weights into the 3090 via the Strata engine's quantized loader. You should verify the total size of the saved .safetensors file against the 24 GB limit and watch the system monitor for the 500 W power draw during long generation runs.
| Metric | Value |
|---|---|
| VRAM usage | 24 GB |
| Reading throughput | 2094 tps |
| Writing throughput | 86.5 tps |
| Power consumption | 500 W |
One core definition: IQ3-S is a weight‑only 3‑bit quantization that preserves activation precision while shrinking model size, making local inference faster and more memory‑efficient on GPUs like the RTX 3090.
If you need fast single‑user generation on a desktop without cloud access, the 3090 with IQ3‑S hits the best balance of speed and memory among available consumer cards.
Monitor temperature and power draw while running the model; the measured 500 W peak and 24 GB VRAM ceiling are stable for standard prompts. If the card throttles, consider undervolting or improving case airflow.
The 3090 lacks Tensor Core support for certain newer formats, so even with IQ3‑S the effective speed is lower than the raw FP16 benchmark. The model still takes several minutes to load from disk.
Corrupted downloads cause crashes on launch. Verify the checksum of the .safetensors file before loading. Use the latest Stratat client update for the best wheel‑on stability.
This local run proves that a 24‑GB VRAM RTX 3090 can handle a sizable language model using the IQ3‑S quantization at roughly 2090 reads per second and 86 reads per second of output write, all within 500 W of power, demonstrating viable edge‑AI use.