You can serve multiple users concurrently on a single Strata instance by tuning the Strata engine's concurrency settings and monitoring hardware limits. This approach requires careful balancing of RAM, VRAM, and CPU load to maintain stability under load.
Why concurrency matters
Serving multiple users simultaneously allows you to maximize the utility of your local hardware investment. Instead of running a single model instance that blocks other requests, you can pipeline operations. This is critical for teams or small offices that rely on local AI for productivity without incurring cloud API costs.
How to configure for concurrent access
First, identify your hardware constraints. The Strata engine must manage memory allocation for each user session. You should check your available system resources before enabling high concurrency. Use your operating system's task manager to monitor RAM and CPU usage in real-time. If the system starts swapping, reduce the number of concurrent users.
Second, adjust the Strata configuration file. Look for the concurrency or batch size parameter. Increase this value gradually while testing with simulated requests. Watch for latency spikes or memory leaks. A stable configuration allows multiple requests to be processed in parallel without crashing the application.
Resource usage under load
The following table illustrates how resource consumption scales with user count. These metrics help you determine the maximum safe concurrency for your specific machine.
| Metric | 1 User | 5 Users | 10 Users |
|---|---|---|---|
| RAM Usage | 4 GB | 8 GB | 12 GB |
| CPU Load | 15% | 45% | 80% |
| Est. Cost | $0.00 | $0.02 | $0.05 |
Note: Costs are estimated using a local electricity price of $0.17/kWh based on typical hardware draw during active inference. Always measure your own hardware to verify these figures.
Core concept
Concurrency is the ability of a system to process multiple requests simultaneously without requiring separate instances for each user, thereby optimizing hardware efficiency.
By following these steps, you can safely serve multiple users on your local Strata instance. Always test incrementally to avoid system overload.