The conclusion is clear: for local inference with Stratat, the RTX 4090 delivers significantly higher throughput thanks to its larger memory bandwidth and more VRAM, while the RTX 5080 trades space for efficiency under the same power envelope.
Why measure power and thermal headroom
Before choosing a card, the user must measure actual consumption on their own machine. On a typical workstation with a Stratat installation, the 4090 draws roughly 450W under peak inference, whereas the 5080 sits nearer 380W. The 5080's 24GB of VRAM versus the 4090's 24GB (both reference designs) means both can host large models, but the 4090's faster GDDR6X memory subsystem yields more tokens per second in practice. Users should verify these figures by monitoring the system's PSU readings and GPU clocks during sustained runs.
How inference speed depends on model size
The Stratat client profiles latency and throughput across common LLM sizes (e.g., 7B, 13B parameters). With a 24GB VRAM budget, both cards accommodate the same model class, yet the 4090 typically shows 15-25% lower time-to-first-token and higher batch throughput. This advantage comes from both clock speed and memory bandwidth. The 5080's newer architecture closes part of the gap, but still lags behind on pure inference benchmarks without external cooling.
| Spec | RTX 4090 | RTX 5080 |
|---|---|---|
| VRAM | 24 GB GDDR6X | 24 GB GDDR6X |
| Power draw (peak inference) | ~450 W | ~380 W |
| Approx. inference tokens/s on 13B model | ~110 tokens/s | ~90 tokens/s |
One core term worth noting: inference is the process of generating output from an already-trained language model, as opposed to training which builds the model. In quotes: "Inference is the process of generating output from an already-trained language model."
Practical guidance for local deployment
Run your own workloads through the Stratat client and record temperature and fan curves; the 5080's lower power can extend quiet hours, but the 4090's superior raw compute remains unmatched for heavy models. Ensure your PSU can comfortably deliver the peak draw, and check that your case airflow accommodates either card. For most users seeking the best local inference performance with Stratat, the RTX 4090 remains the stronger choice despite the newer 5080's efficiency gains.