Issues / #1409
#1409 What is normal prefill tok/s? Suggest to publish this info
closed · @markd89 · 0 コメント · GitHub で見る
本文
Hi, GREAT project! I'm thinking about buying a laptop with GPU to run it. The published tokens/s for decode seem very useful. Can you tell me what a normal prefill tok/s ? Also suggest to include this in the published performance info. I'm trying to understand how Strata + GPU compares with say, a DGX Spark which has only shared/slow RAM. Thanks!
関連リンク
インストール・モデル・リリースへの站内リンク。