反馈 / #862
#862 Got it working on my old 4 Maxwell Titans X, and 2016 dated Supermicro 4029TRT for Qwen 3.8 Flash Next Q8_0
closed · @phalexo · 3 评论 · 去 GitHub 看
说明
I had to use Codex to get this done.
Had to be adapted for Q8_0
Some experts weights went to GPU0, this one does most of the work
Most others to GPU{1,2,3}, these are idle almost all the time
The least used experts went to 330GiB RAM
I don't know if it is interesting to anyone, but the compute on Maxwell Titans is 52, very old.
The rate is up to 11.2 T/S pretty bad but the hardware is a decade old.
Not really certain what I can do with 11 T/S, but an average person reading speed is around 6 T/S
本站相关内容
相关页面的快捷入口。