#672 Better multiGPU support

本文

First of all thanks for this project!

Sorry to bring this subject up, but wouldn't it be usefull to use [this](https://github.com/eddoursul/Strata/tree/custom) project's way of splitting the GPU? I know the fork's owner PR was denied for being too big but his findings are VERY good.

on a pair of 3060 12GB this fork mantains a 1000+ PP/s from first token to 264k while mantaining still a 10/20% tg/s even with all the recent improvements of mainline.

Would be perfect to have that kind of performance!


Also, are concurrent resquests near in the roadmap? Can't wait!