On a desktop equipped with an RTX 4090, running the Coder quantization with 24 GB VRAM yields a strong read speed of 4000 readingTps. The write speed sits at about 109.3 writingTps. The card consumed roughly powerWatts 550 during the test.

Why coder and why this hardware?

The Coder quantization keeps model weights small enough to stay inside the 24 GB of VRAM on the RTX 4090, allowing full local inference without offloading. By choosing a model that runs efficiently on consumer GPUs, you get usable latency while keeping the power draw modest for a desktop rig.

How we measured performance

We used the Strata Engine with the Strata client to run the model locally. The read speed of 4000 readingTps means the model can pull generated tokens fast from VRAM, and the write speed of 109.3 writingTps shows how quickly completions are produced. PowerWatts 550 reflects the sustained draw under load, useful for estimating electricity cost.

MetricValue
VRAM24 GB
readingTps4000
writingTps109.3
powerWatts550

The read and write speeds were captured with a script that streams generated tokens and measures timestamps. You can reproduce the same measurements on your own RTX 4090 by running the same test script and noting the times.

Definition: coder quantization is a compact representation of a model that reduces precision and prunes weights so a single transformer fits into the available VRAM of a consumer GPU.

Practical notes

  • Check that the 24 GB VRAM limit is not exceeded before loading the model.
  • Measure read and write TPS on your own machine to verify the reported performance.
  • Track powerWatts 550 to gauge energy use and expected electricity cost.

Read the full benchmark report for context before deploying.