Issues / #1409

#1409 What is normal prefill tok/s? Suggest to publish this info

closed · @markd89 · 0 评论 · 在 GitHub 查看

Benchmarks

描述

Hi,

GREAT project!

I'm thinking about buying a laptop with GPU to run it.

The published tokens/s for decode seem very useful.

Can you tell me what a normal prefill tok/s ?

Also suggest to include this in the published performance info.

I'm trying to understand how Strata + GPU compares with say, a DGX Spark which has only shared/slow RAM.

Thanks!

站内延伸阅读

链到安装、模型与版本说明,便于 SEO/GEO,非官方 issue 正文。