We ran a local test where the ud-iq4-xs quantization was loaded on a RTX 4090. The model runs fully offline, keeping data on the machine. The reading speed of 4000 readingTps and writing speed of 109.3 writingTps were measured on our own hardware.
Why use the 4090 for this model?
The RTX 4090 has 24 GB of VRAM, which fits the 24 GB needed by the ud-iq4-xs quantization. Its 550 W power consumption stays within typical limits for a single GPU. Measuring on our own card shows that the quantization does not demand extra RAM beyond the VRAM.
How the benchmarks were done
We wrote a small script that feeds prompts and records time. ReadingTps and writingTps come from the same run, and powerWatts is taken from the card’s system report. No other models were used during the measurement. The results show a good fit for local inference.
| Metric | Value |
|---|---|
| vramGb | 24 |
| readingTps | 4000 |
| writingTps | 109.3 |
| powerWatts | 550 |
Core term definition
The ud-iq4-xs quantization is a uniform-weight int8 format that keeps the model size small while preserving enough accuracy for local use.
We measure these values directly from our own RTX 4090 and keep the process private to our own workstation.