#2426 Feature request: better aggregate speeds

本文

I use quite a lot of parallel processing agent workloads (eg memory engine independent of the main agent that processes up to 10+ threads for storage and retrieval), and Starta would be perfect for batching. I know we already have --parallel, but the gains are not as much as in vllm/sglang/exllamav3. Is it possible to reach a higher aggregate speed give if you use a setup that doesn't completely saturate your memory bandwidth?