Pull requests / #295

#295 Pascal build runs (STRATA_EXPERIMENTAL_SM60)

closed · @giostrives · 0 コメント · GitHub で見る

BenchmarksSetup & installMulti-GPU

本文

 I am running this on my machine with 4 × GTX 1080 Ti: I had some issues with the previous commit:

-DSTRATA_EXPERIMENTAL_SM60=ON (01bca8a) builds for Pascal, but the engine still refused the card at startup (src/core/device.cu). The flag now reaches the code and lowers that check to 6.0. setup.py accepts 6.x cards when STRATA_EXPERIMENTAL_SM60=1 and passes the flag to CMake. Nothing changes without the flag.

The speed for the "small" model goes from 27 tok/s to about 35 tok/s on 3 or 4 cards.

The prompt processing speed slows down on multiple GPUs (it is an old machine) and the memory management needs some fixes.

Once this and my other PR for multi-gpu are merged I'll run a second optimisation run for both my machines.

関連リンク

インストール・モデル・リリースへの站内リンク。