#1256 Feature Request: Support for RPCs

closed · @cumal · 1 comments · View on GitHub

Description

First of all, thanks for the effort, with this repo I duplicated TG over default llama.cpp.

As request, I use a rpc server with 2x3060s and maybe it will possible to add rpc connections as llama.cpp.

Thanks!

Related on strata.com

Install Strata
Model sizes & RAM
Release notes

Editorial links to help you install, pick models, or read release notes — not part of the upstream thread.