#2113 Looking forward to support for multiple slots on a single NVIDIA GPU

本文

For users with a single NVIDIA GPU (e.g., RTX 5080), only a single slot is currently supported. Is it possible to support one normal slot plus one short-context slot and provide services through a dedicated API? Many agents have a worker agent and a guardian agent, and the two competing for the same slot causes overall work efficiency to drop sharply. Of course, enabling a single GPU to support 2 slots or more would be the best option.