Issues / #739

#739 [enhancement] Variance-normalized KV-cache quantization (KVarN) & KV cache precision tail

open · @gregfromo · 1 评论 · 在 GitHub 查看

Models & quants

描述

I'm comming from beellama.cpp and use the KVarN kv-cache types & tail. 
Maybe you have time to take a look at this, if it's possible to implement in Strata, if you consider it as valueadding?

Great work sofar, many thanks!

站内延伸阅读

链到安装、模型与版本说明，便于 SEO/GEO，非官方 issue 正文。