To enable a dual-GPU setup in Stratat, you must install compatible NVIDIA or AMD drivers, connect both cards via PCIe, and set the `STRATAT_GPU_IDS` environment variable to include both device indices (e.g., `0,1`) before launching the application. This configuration allows the Stratat engine to distribute model shards across both graphics cards, effectively doubling your available VRAM for larger models without requiring external cloud services.
Why Use Two GPUs?
Running large language models locally often hits a hardware ceiling defined by your graphics card's Video RAM (VRAM). When a model exceeds this limit, the system must offload data to your slower system RAM, causing significant latency. By utilizing two GPUs, you create a larger unified memory pool for the Stratat engine to utilize. This is particularly useful for running 70B parameter models or high-resolution image generation tasks that would otherwise fail to load on a single card.
Step-by-Step Installation
First, ensure your motherboard has sufficient PCIe lanes to support two GPUs without bottlenecking. Most modern motherboards support this configuration natively. Second, install the latest drivers for both cards. For NVIDIA users, this means installing the Studio drivers for stability. For AMD users, ensure ROCm or the appropriate driver stack is active.
1. Verify Hardware Detection
Open your operating system's device manager or terminal. Run the command `nvidia-smi` (for NVIDIA) or `rocm-smi` (for AMD) to verify that both cards are detected. If only one card appears, check your BIOS settings to ensure above 4G decoding is enabled.
2. Configure Stratat Client
Launch the Stratat desktop application. Navigate to the Settings menu and locate the Hardware section. You will see a field for GPU selection. Enter the indices of both cards, typically separated by a comma. For example, if your cards are at index 0 and 1, enter `0,1`.
3. Validate the Setup
Run a small test model through the Stratat engine. Check the logs for a message confirming that both GPUs are active. The engine should show memory usage distributed across both cards.
Hardware Requirements
| Component | Minimum Requirement | Recommended Spec |
|---|---|---|
| GPU Count | 2 NVIDIA RTX 30-series or AMD RX 6000-series | 2 NVIDIA [RTX 4090](/gpus/rtx-4090) or AMD RX 7900 XTX |
| VRAM Total | 24 GB combined | 48 GB combined |
| Power Supply | 750W | 1000W+ |
Core Concept
Dual-GPU setup refers to the configuration where two graphics processing units share the workload of a single AI model, allowing the Stratat engine to access a larger aggregate memory pool for inference.
Frequently Asked Questions
Does Stratat support AMD dual GPUs?
Yes, Stratat supports AMD GPUs. You must ensure both cards are recognized by the system and specify both indices in the settings. The engine uses ROCm to manage the workload across both devices.
Will this speed up inference?
The primary benefit is the ability to run larger models that fit in the combined VRAM. While inference speed may improve due to parallel processing, the main gain is capacity rather than raw token-per-second speed for smaller models.
Can I use mixed GPU brands?
No. Stratat requires both GPUs to be from the same vendor (both NVIDIA or both AMD) to function correctly. Mixing brands will cause the engine to fail to initialize the second card.