反馈 / #862

#862 Got it working on my old 4 Maxwell Titans X, and 2016 dated Supermicro 4029TRT for Qwen 3.8 Flash Next Q8_0

closed · @phalexo · 3 评论 · 去 GitHub 看

说明

I had to use Codex to get this done. 
Had to be adapted for Q8_0
Some experts weights went to GPU0, this one does most of the work
Most others to GPU{1,2,3}, these are idle almost all the time
The least used experts went to 330GiB RAM

I don't know if it is interesting to anyone, but the compute on Maxwell Titans is 52, very old.
The rate is up to 11.2 T/S pretty bad but the hardware is a decade old. 

Not really certain what I can do with 11 T/S, but an average person reading speed is around 6 T/S

本站相关内容

相关页面的快捷入口。