Pull requests / #295

#295 Pascal build runs (STRATA_EXPERIMENTAL_SM60)

closed · @giostrives · 0 评论 · 在 GitHub 查看

BenchmarksSetup & installMulti-GPU

描述

 I am running this on my machine with 4 × GTX 1080 Ti: I had some issues with the previous commit:

-DSTRATA_EXPERIMENTAL_SM60=ON (01bca8a) builds for Pascal, but the engine still refused the card at startup (src/core/device.cu). The flag now reaches the code and lowers that check to 6.0. setup.py accepts 6.x cards when STRATA_EXPERIMENTAL_SM60=1 and passes the flag to CMake. Nothing changes without the flag.

The speed for the "small" model goes from 27 tok/s to about 35 tok/s on 3 or 4 cards.

The prompt processing speed slows down on multiple GPUs (it is an old machine) and the memory management needs some fixes.

Once this and my other PR for multi-gpu are merged I'll run a second optimisation run for both my machines.

站内延伸阅读

链到安装、模型与版本说明,便于 SEO/GEO,非官方 issue 正文。