#2113 Looking forward to support for multiple slots on a single NVIDIA GPU

open · @jiayzhangithub · 0 comments · View on GitHub

Description

For users with a single NVIDIA GPU (e.g., RTX 5080), only a single slot is currently supported. Is it possible to support one normal slot plus one short-context slot and provide services through a dedicated API? Many agents have a worker agent and a guardian agent, and the two competing for the same slot causes overall work efficiency to drop sharply. Of course, enabling a single GPU to support 2 slots or more would be the best option.

Related on strata.com

Install Strata
Model sizes & RAM
Release notes

Editorial links to help you install, pick models, or read release notes — not part of the upstream thread.